I can't get to execute symfony command in bash script when I run it in cron.
When I execute the .sh script by hand everything is working fine.
in my bash file the command is executed like this:
/usr/bin/php -q /var/www/pww24/bin/console pww24:import asari $office > /dev/null
I run the scripts from root, the cron is set to root as well. For the test i set files permissions to 777 and added +x for execution.
the bash script executes fine. It acts like it's skipping the command but from logs i can see that the code is executed
It turned out that symfony system variables that I have stored on server are not enough. When you start to execute the command from command line its fine, but when using Cron you need them in .env file. Turned out that in the proces of countinous integrations I only got .env.dist file and I've to make the .env file anyways.
Additionaly I've added two lines to cron:
PATH=~/bin:/usr/bin/:/bin
SHELL=/bin/bash
and run my command like this from the bash file:
sudo /usr/bin/php -q /var/www/pww24/bin/console pww24:import asari $office > /dev/null
I am new to scripting part. I have a cron job running which trigger a shell script file. Inside the file I have below command to remove the file
pbrun rm -f {some file location}
I have to use pbrun because file is having the required permission for root (which I don't have) and for pbrun. Manually running the same command deletes the desired file but running it through script it is not working.
When I want to compress a folder manually I use the command:
tar -zcvf $dirBackup-$actualTime.tgz $dirBackup;
And it works perfectly. Also when I decompress the archive the result is correct.
But I have a problem when I run the same command from crontab: the resulting tgz only appears to contain an empty file.
crontab -l>> /home/user/Desktop/test/temp;
echo "$min $hour * * * tar -zcvf $dirBackup-$actualTime.tgz $dirBackup">> temp;
crontab temp;
Do I need to change anything in the crontab expression? Do I have to be root to run crontab? The file temp is created correctly, and located in the right folder.
Thanks a lot for your attention!
Depending on your system, crontab jobs may or may not be run in your $HOME directory (in fact they are typically run either in / or in /var/cron). Add a cd $HOME; before the tar command.
You need to set the value of the variables in the echo command you are using to add entry into the cron job. A sample script should look like this:
#!/bin/bash
min=30
hour=17
dirBackup=/home/user/Desktop
actualTime=30-04-2015
crontab -l >> /home/user/temp
echo "$min $hour * * * tar -zcvf $dirBackup-$actualTime.tgz $dirBackup 2>&1 | tee /tmp/cron.log">> /home/user/temp;
crontab /home/user/temp;
The resulting tar file will be created in the user's HOME directory (user with which the cron is run).
To avoid getting into user permission issues, you can specify the user with which the cron is to be run using the following command:
crontab -u <username> <cron_file>
I have a command in my shell script (increment.sh):
TMP_FILE=/tmp/sbg_clickstream.tmp
hive -e "select * from $HIVE_TMP_TABLE;" > $TMP_FILE
I am getting an error on the 2nd line:
/tmp/sbg_clickstream.tmp: Permission denied
Error: Error occured while opening data file
Error: Load Failed, records not inserted.
Load failed (exit code 1)
I tried chmod 750 /tmp/sbg_clickstream.tmp and ran the script again but I am still getting the same error.
I am new to shell scripting and I thought the dir/file is created when TMP_FILE=/tmp/sbg_clickstream.tmp. However after the test above, concluded it does not.
I think the dir/file is created during this line:
hive -e "select * from $HIVE_TMP_TABLE;" > $TMP_FILE
How can I change the permissions while the query is populating and creating the file?
Two questions:
What user is running the script?
Who owns the file /tmp/sbg_clickstream.tmp?
If the user running the script owns the file, then the problem may be with the shell setting itself.
Type set -o from the command line, and check the value of clobber or noclobber. If noclobber is set to on or clobber is set off, then you cannot overwrite files via the redirection. You'll need to set or unset clobber/noclobber.
If your shell has a value for clobber (like the Kornshell), you need to do this:
$ set -o clobber # Turns clobber on
If your shell has a value for noclobber (like BASH), you need to do this:
$ set +o noclobber # Turns noclobber off
Yes, the -o/+o parameters seem backwards.
Hint:
Instead of using the same file name over and over, try this:
TMP_FILE=/tmp/sbg_clickstream.$$.tmp
hive -e "select * from $HIVE_TMP_TABLE;" > $TMP_FILE
The $$ represents the pid, and thus changes all the time. When the system reboots, it should clean out the /tmp directory. Otherwise the PID climbs and shouldn't repeat. This way, each run generates a new temp file, and you don't have to worry about clobbering or not.
Even Better Hint
See if your system has a mktemp command. This will generate a unique temporary file name:
TMP_FILE=$(mktemp -t sbg_clickstream.XXXXX)
echo "The tempfile is '$TMP_FILE'
hive -e "select * from $HIVE_TMP_TABLE;" > $TMP_FILE
This may echo something like this:
The tempfile is /tmp/sbg_clickstream.Ds23d
mktemp is guaranteed to create a valid and unique temporary file name.
chmod/chown is your friend but you need to ensure that the user/group running the shell script has permission to write to the /tmp directory.
What is the output of the following:
ls -ld /tmp
whoami
groups
This question already has answers here:
CronJob not running
(19 answers)
Closed last month.
I have a cron job that I want to execute every 5 minutes:
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /scr_temp/scheduleSpider.sh
In /var/spool/cron/crontabs/root
The cron should execute a shell script:
#!/bin/sh
if [ ! -f "sync.txt" ]; then
touch "sync.txt"
chmod 777 /scr_temp
curl someLink
fi
That works fine from command line but not from cron. However the cron itself is startet but the script does not start.
I read about the path problem but I dont really understand it. I setup a cron that writes some env data to a file. This is the output:
HOME=/root
LOGNAME=root
PATH=/usr/bin:/bin
SHELL=/bin/sh
If I execute the env command in command line I get following output for PATH
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
What path do I have to set in my shell script?
Your $PATH is fine; leave it alone. On Ubuntu, all the commands you're invoking (touch, chmod, curl) are in /bin and/or /usr/bin.
How did you set up the cron job? Did you run crontab some-file as root?
It seems that /etc/crontab is the usual mechanism for running cron commands as root. On my Ubuntu system, sudo crontab -l says no crontab for root. Running crontab as root, as you would for any non-root account, should be ok, but you might consider using /etc/crontab instead. Note that it uses a different syntax than an ordinary crontab, as explained in the comments at the top of /etc/crontab:
$ head -5 /etc/crontab
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.
Run sudo crontab -l. Does it show your command?
Temporarily modify your script so it always produces some visible output. For example, add the following right after the #!/bin/sh:
echo "Running scheduleSpider.sh at \`date\`" >> /tmp/scheduleSpider.sh.log
and see what's in /tmp/scheduleSpider.sh.log after a few minutes. (You can set the command to run every minute so you don't have to wait as long for results.) If that works (it should), you can add more echo commands to your script to see in detail what it's doing.
It looks like your script is designed to run only once; it creates the sync.txt file to prevent it from running again. That could be the root (ahem) of your problem. What that your intent? Did you mean to delete sync.txt after running the command, and just forgot to do it?
root's home directory on Ubuntu is /root. The first time your script runs, it should create /root/sync.txt. Does that file exist? If so, how old is it?
Note that curl someLink (assuming someLink is a valid URL) will just dump the content from the specified link to standard output. Was that your intent (it will show up as e-mail to root? Or did you just not show us the entire command?
First: you can substitute the first field with */5 (see man 5 crontab)
Second: have cron mail the output to your email address by entering MAILTO=your#email.address in your crontab. If the script has any output, it'll be mailed. Instead of that, you may have a local mailbox in which you can find the cron output (usually $MAIL).
A better syntax for you CRON is
*/5 * * * * /scr_temp/scheduleSpider.sh
Also, check the authority of your scheduleSpider.sh file. Cron runs under a different user than the one you are likely executing your program interactively, so it may be that cron does not have authority. Try chmod 777 for now, just to check.
I suggest to:
check that /scr_temp/scheduleSpider.sh has executable bit
set PATH properly inside your script or use absolute path to command (/bin/touch instead of touch)
specify absolute path to sync.txt file (or calculate it relatively to script)
Have you added the comand via crontab -e or just by editing the crontab file? You should use crontab -e to get it correctly updated.
Set the working directory in the cron script, it probably doesn't execute the things where you think it should.
You should add /bin/sh before the absolute path of your script.
*/5 * * * * /bin/sh /scr_temp/scheduleSpider.sh