On startup I manually run source /usr/local/etc/credentials.sh.
I've made a script /usr/local/etc/rc.d/load_credentials to run command for me at start-up. However the script does not succeed:
$ /usr/local/etc/rc.d/load_credentials start
eval: source: not found
load_credentials
The script is based on the example in rc-scripting docs.
#!/bin/sh
. /etc/rc.subr
name="load_credentials"
start_cmd="${name}_start"
stop_cmd=":"
load_credentials_start()
{
source /usr/local/etc/psql_creds.sh
}
load_rc_config $name
run_rc_command "$1"
credentials.sh
export DB_USER="JimmyJohn"
export DB_PASS="password123"
EDITs
Moved files as advised by #Michael-O
Based on your comment
will the credentials be available for crontasks running from my user?
it would seem you wish this to work in crontasks.
Here is a sample cronjob that I tested working and it does indeed work in crontasks.
Make a file in /usr/local/bin/crondo.sh
# chmod +x /usr/local/bin/crondo.sh
# cat /usr/local/bin/crondo.sh
#!/bin/sh
. /usr/local/etc/psql_creds.sh
echo $DB_USER
Add this line to crontab with crontab -e
# Run 22 minutes after every hour.
22 * * * * /usr/local/bin/crondo.sh > /tmp/out
Related
UPDATE After writing out the cron process per Gordon's comment on this post, it seems that Cron cannot find the script. Any guidance on this would be appreciated.
>>/tmp/HRsnapshot.log 2>&1
/bin/sh: 1: /home/user/Scripts/HRsnapshot.sh: not found
/bin/sh: 1: /home/user/Scripts/HRsnapshot.sh: not found
/bin/sh: 1: /home/user/Scripts/HRsnapshot.sh: not found
I need some assistance in understanding why a cronjob is not executing a script that runs as designed when executed manually from the user account.
Currently, the configuration is as such.
Crontab
#Download HR File from Google and push it to HR Server via SFTP
38 11 * * * BASH_ENV=/home/user/Scripts/preload.sh /home/user/Scripts/HRsnapshot.sh
We are loading user env variables in the preload.sh script
#!/bin/bash
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
. /etc/profile
. ~/.bashrc
export GAMCFGDIR="/home/user2/GAMConfig"
The Bash profile has some callouts in it to an application named GAM
#Setting for Gam Config
export GAMCFGDIR="/home/user2/GAMConfig"
export PATH
alias gam="/home/user2/bin/gamadv-xtd3/gam"
The primary script is designed to download a file using GAM (Python app that simplifies API calls to Google Cloud for a Google Workspace env)
#!/bin/bash
cd /home/user/Scripts
/home/user2/bin/gamadv-xtd3/gam user 'hellodolly#conso.org' get drivefile id '1S4tkzS2lIj360-WFhTy3FBm6kZ7hsL4PvO7PdikffiidI' format 'csv' targetfolder /home/user/SFTP/daily_export/ targetname 'FY2022_HR_Form.csv' overwrite
sshpass -p '*jndn4Kz#8Rwrj' sftp GoogleSFTP#10.0.0.0<<EOF
cd /D:/SFTPRoot/Google_Workflows
put /home/user/SFTP/daily_export/FY2022_HR_Form.csv
exit
EOF
Cronlog
Oct 15 13:07:01 JUMPBOX CRON[5431]: (user) CMD (BASH_ENV=/home/user/Scripts/preload.sh /home/user/Scripts/HRsnapshot.sh)
Here is what the user's env looks like
SHELL=/bin/bash
SUDO_GID=1000
SUDO_COMMAND=/usr/bin/su user
SUDO_USER=user2
PWD=/home/user/Scripts
LOGNAME=user
HOME=/home/user
LANG=en_US.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
LC_TERMINAL=iTerm2
GAMCFGDIR=/home/user2/GAMConfig
LESSCLOSE=/usr/bin/lesspipe %s %s
TERM=xterm-256color
LESSOPEN=| /usr/bin/lesspipe %s
USER=user
LC_TERMINAL_VERSION=3.4.16
SHLVL=1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
SUDO_UID=1000
MAIL=/var/mail/user
OLDPWD=/home/user2
All user names, IP addresses, Emails Addresses, and Passwords have been obfuscated
Try changing your preload script, export your PATH after sourcing the files.
#!/bin/bash
. /etc/profile
. ~/.bashrc
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
export GAMCFGDIR="/home/user2/GAMConfig"
When sshpass is in /usr/local/bin you don't want the path changed by /etc/profile or .bashrc.
Thanks, everyone for your help.
Strangest thing. There is something invisible in the copied text for the cronjob that is causing the error.
38 11 * * * BASH_ENV=/home/dpmacrina/Scripts/preload.sh /home/dpmacrina/Scripts/HRsnapshot.sh
03 06 * * * BASH_ENV=/home/dpmacrina/Scripts/preload.sh /home/dpmacrina/Scripts/HRSnapshot.sh
These are the two lines that are in the crontab. The top is what was failing, the bottom is what I typed in new. The bottom works.
I wanna run a script using crontab in every minute. The script is located at /Users/robin47/Desktop/script/demo.sh. As a side note, I give the execute permission of demo.sh file with chmod +x demo.sh command.
But it doesn't execute the script. Here is the process I'm following:
Run crontab -e from the terminal.
robin47#mac-pro ~ % crontab -e
this open a file(crontab.xxxxxxx) in nano which is located under tmp folder, as an example File: /tmp/crontab.V4DcXEJU8g.
This is my crontab.xxxxxxx file:
SHELL=/bin/bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin
* * * * * /bin/bash /Users/robin47/Desktop/script/demo.sh
Get from here, this doesn't execute my demo.sh script.
If I run sh command from terminal like sh demo.sh, it works. And if I directly echo(or run any command like touch, mkdir) in crontab like this
SHELL=/bin/bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin
* * * * * echo 'hello' >> /Users/robin47/Desktop/script/text.txt
It also works. But doesn't run the script using crontab. I appreciate your help, Thanks.
This is my demo.sh file:
#!/bin/sh
echo "hello world" >> /Users/robin47/Desktop/script/text.txt
refer below site for cron entries , also check your script have executable permission
chmod oug+x demo.sh
https://crontab.guru/
Creating a script to pass to a few different people and ran into an env problem. The script wouldn't run unless I supplied it with $PATH, $HOME, and $GOPATH at the beginning of the file. Like so:
HOME=/home/Hustlin
PATH=/home/Hustlin/bin:/home/Hustlin/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin:/bin:/home/Hustlin/go/bin
export GOPATH=$HOME/go
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
This is not advantageous when trying to pass the script around and each person has to set these variables themselves. This file would rarely be run by the User and would most often be run via crontab.
I would love to hear a better way of coding this so I'm not asking everyone I send the script to update these variables.
Thank you all in advance!!!
EDIT
The script is being run via crontab with no special permissions.
1,16,31,46 * * * * /home/Hustlin/directory1/super_cool_script.sh
Here is the script I am running:
#!/bin/bash
# TODO Manually put your $PATH and $HOME here.
PATH=/home/Hustlin/bin:/home/Hustlin/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin:/bin:/home/Hustlin/go/bin
HOME=/home/Hustlin
export GOPATH=$HOME/go
export PATH=$PATH:$GOROOT/bin:$GOPATH/bin
# Field1
field1="foo"
# Welcome message.
echo Starting the update process...
# Deposit directory.
mkdir -p $HOME/directory1/sub1/data/body
mkdir -p $HOME/directory1/sub2/system
# Run command
program1 command1
# Run longer command.
program1 command2 field1
sleep 3
program1 command3 -o $HOME/directory1/sub1/data $field1
sleep 1
# Unzip and discard unnecessary files.
unzip $HOME/directory1/sub1/data/$field1 -d $HOME/directory1/sub1/data
rm $HOME/directory1/sub1/data/bar.yaml $HOME/dircetory1/sub1/data/char.txt
rm $HOME/directory1/sub1/data/$field1.zip
# Rename
mv $HOME/directory1/sub1/data/body.json $HOME/directory1/sub1/data/body/$(date -d '1 hour ago' +%d-%m-%Y_%H).json
echo Process complete.
I changed most of the program and command names for privacy. What I did post still represents what is being done and how the files are being moved.
The issue is crontab, not the script.
When you run the script on your terminal, you are logged in a session with all environment variables set, so the script can use it.
But when you run it from crontab it an "empty" session, so it does not have any environment variable set, it doesn't even know about your user.
Run the script on crontab like this:.
su --login Hustlin /home/Hustlin/directory1/super_cool_script.sh
Check this documentation.
http://man7.org/linux/man-pages/man1/su.1.html
bash -l -c /path/to/script will make bash execute all .bashrc and .profile files first, so it will have HOME and PATH variables set.
Hi guys am trying to automate the backup of snapshots for my ec2 volumes on Amazon. I am following the ec2-automate-backup script by Collin Johnson
If run the command on command line it is creating the snapshot (working):
ubuntu#linuxserver:/usr/local/ec2/scripts$ sudo ./ec2-automate-backup.sh -s tag -t "Backup,Values=true" -c ./cron-primer.sh -r "eu-west-1"
For testing purposes if i create a crontab its not working
0 10 * * * ubuntu /usr/local/ec2/scripts/ec2-automate-backup.sh -s tag -t "Backup,Values=true" -c /usr/local/ec2/scripts/cron-primer.sh -r "eu-west-1"
Where is my problem here am running the script on ubuntu 14.04 - Amazon?
In crontab file, to execute a shell script you can use one of the following approach:
1. Call the shell script direcly, i.e.
0 10 * * * /path/to/script.sh
where the script.sh should be made executable.
2. Execute the script by sh utility, i.e.
0 10 * * * sh /path/to/script.sh
here the script.sh need not be made executable.
Now, if in your case, you need to go to a specific path and then execute script, then :
Either provide the full path of the script in crontab file directly, or
Enclose the execution commands in other shell file, and execute the enclosing file from cron.
There are two possibilites:
You need root access to run the script. You can solve this by modifying root's crontab:
sudo crontab -e
See How to run a cron job using the sudo command
You need to be in the same directory as the script to execute it
0 10 * * * ubuntu cd /usr/local/ec2/scripts && ./ec2-automate-backup.sh -s tag -t "Backup,Values=true" -c ./cron-primer.sh -r "eu-west-1"
See What is the 'working directory' when cron executes a job
I have pretty simple command which is working fine standalone as a command or bash script but not when I put it in crontab
40 05 * * * bash /root/scripts/direct.sh >> /root/cron.log
which has following line
PATH=$PATH:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
SHELL=/bin/sh PATH=/bin:/sbin:/usr/bin:/usr/sbin:/root/
# Mongo Backup
docker exec -it mongodb mongodump -d meteor -o /dump/
I tried to change the url of script to /usr/bin/scirpts/ no luck
I even tried to run script directly in cron
26 08 * * * docker exec -it mongodb mongodump -d meteor -o /dump/ >> /root/cron.log
with no luck, any help appreciated.
EDIT
I don't see any errors in /root/cron.log file either
Your docker exec command says it needs "pseudo terminal and runs in interactive mode" (-it flags) while cron doesn't attach to any TTYs.
Try changing your docker exec command to this and see if that works?
docker exec mongodb mongodump -d meteor -o /dump/
for what it's worth I had this exact same problem. Fixing your PATH, changing permissions, and making sure you are running as the appropriate docker user are all good things, but that's not enough. It's going to continue failing because you are using "docker exec -it", which tells docker to use an interactive shell. Change it to "docker exec -t" and it'll work fine. There will be no log output anywhere telling you this, however. Enjoy!
cron debugging
1. /var/log or sendmail
As crond work as a daemon, without ability of failing, execution is more important than logging. Then by default, if something goes wrong, cron will send a mail to $USER#localhost reporting script output and errors.
Have a look at /var/mail or /var/spool/mail for some mails, maybe
and at /etc/aliases to see where root's mail are sents.
2. crond and $PATH
When you run a command by cron, have care that $PATH is user's default path and not root default path (ie no */sbin and other reserved path to super user tools).
For this, the simplier way is to print your default path in the environment where everything run fine:
echo $PATH
or patch your script from command line:
sed -e "2aPATH='$PATH'" -i /root/scripts/direct.sh
This will add current $PATH initializer at line 2 in your script.
Or this, will whipe from your script all other PATH=:
sed -e "s/PATH=[^ ]*\( \|$\)/\1/;2aPATH='$PATH'" -i /root/scripts/direct.sh
3. Force logging
Add at top of your script:
exec 1>/tmp/cronlog-$$.log
exec 2>/tmp/cronlog-$$.err
Try this:
sed -e '1a\\nexec 1>/tmp/cronlog-$$.log\nexec 2>/tmp/cronlog-$$.err' -i ~/scripts/direct.sh
Finalized script could look like:
#!/bin/bash
# uncomment two following lines to force log to /tmp
# exec 1>/tmp/cronlog-$$.log
# exec 2>/tmp/cronlog-$$.err
PATH='....' # copied from terminal console!
docker exec -it mongodb mongodump -d meteor -o /dump/
Executable flag
If you run your script by
40 05 * * * bash /root/scripts/direct.sh
no executable flag are required, but you must add them:
chmod +x ~/scripts/direct.sh
if you want to run:
40 05 * * * /root/scripts/direct.sh
1) Make sure this task is in the root user's crontab - it's probably the case but you didn't write it explicitly
2) cron may be unable to find bash. I would remove it and call directly your script after making it executable:
chmod 755 /root/scripts/direct.sh
and then set your crontab entry as 40 05 * * * /root/scripts/direct.sh 2>&1 >> /root/cron.log
If it's still not working, then you should have some useful output in /root/cron.log
Are you sure your script is running? Add an other command like touch /tmp/cronok before the docker exec call.
Don't forget that the crontab needs a newline at the end. Use crontab -e to edit it.
Restart the cron service and check the logs (grep -i cron /var/log/syslog).
If your OS is redhat/centos/fedora, you should try with the username (root) between the frequency and the command.
Check your mails with the mail command.
Check the crontab permissions. chmod 644 /etc/crontab.
Maybe you just don't want to reinvent the wheel.
Here's a few things I'd change-- first, capture STDERR along with STDOUT and remove the shell specification in cron-- use #! in your script instead.
40 05 * * * /root/scripts/direct.sh &>> /root/cron.log
Next, you are setting your PATH in the reverse order, and you are missing your shbang. I have no idea why you are defining SHELL as /bin/sh, when you are running bash, instead of dash. Change your script to this.
#!/usr/bin/env bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/root
PATH=$PATH:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
# Mongo Backup
docker exec -it mongodb mongodump -d meteor -o /dump/
See if that yields something better to work with.