I have a cron job to run a sed shell called sedcmd.sh to pre-process some json data. When i am in the proper directory I manually run it with
. ./sedcmd.sh
And it works. The shell itself works fine.
for reference one of the commands inside looks like
sed -i '/^\s*$/d' /home/school/Desktop/Programs/rawjsondata.txt
my cronjob looks like
5 * * * * . ./home/school/Desktop/Programs/sedcmd.sh
I get the error "No such file or directory found". What am I doing wrong. I've triple checked for any random spelling errors. I also can't seem to run sedcmd.sh from any other directory even when i give the entire file path, so its definitely something I'm doing wrong. My thoughts for solutions are either
I should add sedcmd.sh to my $PATH or bashrc so i can call it from anywhere. Which I don't know how to do.
OR
figure out how to call it correctly from crontab. Which I also dont know how to do.
When you are running a script from terminal, you do:
./script_name.sh but to execute the same script from crontab you do something like 5 * * * * /path/to/script/script_name.sh
As Sam has got this answer from the comments,posting the answer as community wiki.
Related
I've created a bash file that queries my database and then updates some tables.
When I run it manually everything goes smoothly but when I run it with a cronjob it runs the first query and then stops before it goes into a loop.
After looking into it on the net I found a few things that may be the issue but from my side everything looks in order.
So what I did:
Checked if #!/bin/bash is included in my bash at the start and it is.
Checked that the path is correct in the cronjob. My cronjob below
0-59/5 * * * * cd /path/path2/bashLocation/; ./bash.sh
The loop is in the format of
for ID in ${IDS//,/ }
do
...do something
done
This works fine tested manually. My IDS are in string format that why I split it with //,/.(Works fine)
I log all outputs in a log file but it doesn't show any error.
Has anyone encountered this issue before or has any ideas how to fix the issue?
If the command you are running in cron has percent signs ('%'), they need to be escaped with a backslash. I've been bitten by this. From the manpage: "Percent-signs (%) in the command, unless escaped with backslash () ..."
The $PATH variable may be different when run from cron. Try putting something like this at the beginning of your script: export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Try running bash explicitly, i.e. rather than ./bash.sh in crontab, try /bin/bash bash.sh
I don't know how helpful this may be to some people but I noticed when I printenv shell in my logs it printed that it was bin/sh even if I define it at the top of my script and run it as a bash file.
So what I did was changed all parts of my code that where not supported by shell and my conjob works fine.
So I assume that conjob does not support bash files. (Didn't find anything on the internet about this.)
Why it runs in /bin/sh I don't know.
Hope someone finds this helpful.
I have a Python3 program that I need to execute 2 minutes past every hour.
Unable to get python to execute directly from cron, I’ve written a shell script with the commands in it.
The crontab to execute this shell script is this:
*/2 * * * * bash /home/john/PYTHONS/MASTER.sh
The shell script is simple. I’ve tried various versions, including this:
#!/bin/bash
cd /home/john/PYTHONS/
python3 ~/PYTHONS/COMPLEXPYTHON.py
python3 ~/PYTHONS/SIMPLEPYTHON.py
and including this:
#!/bin/bash
cd /home/john/PYTHONS/GDAXDEV/VPUMP
python3 $HOME/PYTHONS/COMPLEXPYTHON.py
python3 $HOME/PYTHONS/SIMPLEPYTHON.py
Here’s the issue:
If I execute the shell script from the command line BOTH python scripts run just fine.
If I execute the shell script from cron, ONLY THE SECOND python script runs. The first one just doesn’t run at all. And, I can’t seem to find out why, or how to fix it.
Any suggestions or help would be greatly appreciated.
I see questions about this issue all around, and no single one of them solved the issue for me. Several hours of testing every combination of variables lead me to a solution that’s reliable.
I’m no longer using a shell script, but invoking python directly in cron.
First, we did this:
$ which python3
/home/john/[path to python]/bin/python3
So, we added the shebang in every one of the python files in the project:
#!/home/john/[path to python]/bin/python3
And, in cron we used the full path to that python3, as well as the full path to the WORKING.py file we wanted to run. We want the thing to run 5 minutes past every hour so:
5 * * * * /home/john/[path to python]/bin/python3 /home/john/[path to dir]/WORKING.py
However, the log files this produced were being placed in /home/john/, and I wanted them in the specific directory. Also, there are a load of additional python files in there – with the same names as python files in nearby directories – so I wanted to be certain that any call gets the right one.
So, I added a CD command at the beginning of the cron command:
5 * * * * cd /home/john/[path to dir]/ && /home/john/[path to python]/bin/python3 /home/john/path to dir]/WORKING.py
It’s been running reliably ever since.
I’m grateful for the resources at stackoverflow and the great and fast help I’ve received in this community, and I hope this entry helps save someone else a whole afternoon of frustration.
I have a shell script like getlist.sh it has a below hive command inside it.
`hive -e "select * from databasename.tablename where name = 'myname';" > /home/testuser/list.tsv`
when run the getlist.sh manually it works fine, when I schedule it using cron it creates the file but with no record in it.
Can someone correct me, couldn't figure out where I'm going wrong.
I fixed it by adding #!/bin/bash instead of #!/bin/sh in getlist.sh script. it did the trick.
I am using an EC2 instance, crontab, and slack-cleaner to delete all Slack messages older than 48 hours. To do this, I created delete_slack.sh (I've deleted my slack api token):
for CHANNEL in random general
do
slack-cleaner --token <MY TOKEN> --message --channel $CHANNEL --user "*" --before $(date -d '48 hour ago' "+%Y%m%d") --perform
done
Then I created a crontab line to run it every minute (once it works I'll change the timing to once a day) and had cron spit out the results to a log file:
* * * * * /home/ubuntu/delete_slack/delete_slack.sh >> /var/log/delete_slack.log 2>&1
To test, I ran sh /home/ubuntu/delete_slack/delete_slack.sh >> /var/log/delete_slack.log 2>&1 in the shell and it works fine. However, when I let the crontab run I get an error in the log file:
/home/ubuntu/delete_slack/delete_slack.sh: 3: /home/ubuntu/delete_slack/delete_slack.sh: slack-cleaner: not found
Any ideas? I've been banging my head against this all afternoon.
Sounds like the PATH you get via cron and the PATH you get through your login are different.
Either set the PATH in your script or use the absolute path to slack-cleaner
The PATH tells the shell which directories to search for executables (including scripts). You can echo $PATH to compare your path to the one cron gives and confirm that this is the issue.
If using the absolute path works, that is simplest, but if slack-cleaner uses other exes itself, setting the path may be better.
If you want to go the "modify PATH" method then you want to append the correct path to existing PATH and not completely overwrite it. i.e. export PATH=$PATH:/path/to/slack-cleaner-dir. You can always use which slack-cleaner to find out the correct path. NOTE: you want the directory without "slack-cleaner" appended to the end.
ALWAYS use full path in crons and you'll save a lot of time.
If you don't like export PATH=... then just use /path/to/slack-cleaner-dir instead.
Just load your profile before running the command to be in the exact same situation as when you launch it from your shell :
* * * * * . ~/.profile;/home/ubuntu/delete_slack/delete_slack.sh >> /var/log/delete_slack.log 2>&1
As I read that you're a bit new to this, here are just some more explanations about the profile :
The profile is a file loaded automatically when you connect in shell with your user.
The file is hidden in your home directory, to see it, you can launch :
ls -la ~
If you're in bash, the file will be named .bash_profile, if you're in shell or ksh, it will be named .profile
Hope it helped !
I have a cronjob set every 3 minutes to run a bash script. Now, I wish to separate the output and the errors to two separate files. I did read the answer given in this AskUbuntu question and tried */3 * * * * ./script.sh>>output.log 2>>errors.log (>>, since I'm creating the file in case it doesn't exist), but this seems to redirect the output again to errors.log. What am I doing wrong?
That should work just fine.
Perhaps your problem is that you are not using absolute paths for the script and the log files?