Is there anything like lint for crontab? I'd like to know that i've got all my spaces and stars sorted out without waiting for something to not work.
There's a Python linter for crons. See chkcrontab project
You can install it via pip:
pip3 install chkcrontab
Example usage:
chkcrontab /etc/cron.d/power-schedule
Checking correctness of /etc/cron.d/power-schedule
E: 15: 0 12 * foo * * root echo hi
e: FIELD_VALUE_ERROR: foo is not valid for field "month" (foo)
e: INVALID_USER: Invalid username "*"
E: There were 2 errors and 0 warnings.
I've found CronWTF to be incredibly helpful when writing crontabs - it translates your stars and commands into something more human friendly, to make it easier to read strange cron jobs.
Better yet, because it's all javascript you can run it locally, and noone need know about your top sekrit cron jobs.
Another alternative if you code ruby is to use the whenever gem - you use a sample ruby file called schedule.rb to parse, and generate crontabs from like so:
every 10.minutes do
command "/usr/bin/my_great_command"
end
Will give you a crontab entry of
0,10,20,30,40,50 * * * * /usr/bin/my_great_command
And this one here:
every 2.days, :at => '4:30am' do
command "/usr/bin/my_great_command"
end
Will give you:
30 4 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31 * * /usr/bin/my_great_command
I don't think you need a lint for crontab. There's 5 fields that are space separated then a space then the command to run and its args finish off the line.
Also, on Ubuntu at least, crontab won't let you save a bum file. I just tried a few things and it barfed on all of them. I guess that means that crontab is its own 'lint for cron'.
It might be a bit off, but an easy way would be to just load it with a graphical crontab editor like kcron or gcrontab. If you need to call it in a script, this question is about how to do it in php.
I'm not sure if this is the sort of thing you're looking for, but it makes writing crontabs really easy by showing you exactly what you're setting the schedule to:
https://crontab.guru/
You can try shell script named 48-verifycron from Wicked Cool Shell Scripts, 2nd Edition,
if you can't access python and pip to use chkcrontab
Related
I have a Python3 program that I need to execute 2 minutes past every hour.
Unable to get python to execute directly from cron, I’ve written a shell script with the commands in it.
The crontab to execute this shell script is this:
*/2 * * * * bash /home/john/PYTHONS/MASTER.sh
The shell script is simple. I’ve tried various versions, including this:
#!/bin/bash
cd /home/john/PYTHONS/
python3 ~/PYTHONS/COMPLEXPYTHON.py
python3 ~/PYTHONS/SIMPLEPYTHON.py
and including this:
#!/bin/bash
cd /home/john/PYTHONS/GDAXDEV/VPUMP
python3 $HOME/PYTHONS/COMPLEXPYTHON.py
python3 $HOME/PYTHONS/SIMPLEPYTHON.py
Here’s the issue:
If I execute the shell script from the command line BOTH python scripts run just fine.
If I execute the shell script from cron, ONLY THE SECOND python script runs. The first one just doesn’t run at all. And, I can’t seem to find out why, or how to fix it.
Any suggestions or help would be greatly appreciated.
I see questions about this issue all around, and no single one of them solved the issue for me. Several hours of testing every combination of variables lead me to a solution that’s reliable.
I’m no longer using a shell script, but invoking python directly in cron.
First, we did this:
$ which python3
/home/john/[path to python]/bin/python3
So, we added the shebang in every one of the python files in the project:
#!/home/john/[path to python]/bin/python3
And, in cron we used the full path to that python3, as well as the full path to the WORKING.py file we wanted to run. We want the thing to run 5 minutes past every hour so:
5 * * * * /home/john/[path to python]/bin/python3 /home/john/[path to dir]/WORKING.py
However, the log files this produced were being placed in /home/john/, and I wanted them in the specific directory. Also, there are a load of additional python files in there – with the same names as python files in nearby directories – so I wanted to be certain that any call gets the right one.
So, I added a CD command at the beginning of the cron command:
5 * * * * cd /home/john/[path to dir]/ && /home/john/[path to python]/bin/python3 /home/john/path to dir]/WORKING.py
It’s been running reliably ever since.
I’m grateful for the resources at stackoverflow and the great and fast help I’ve received in this community, and I hope this entry helps save someone else a whole afternoon of frustration.
I am looking to automate an interactive install process with ansible. This install does not have a silent install option or does not take command line arguments for the interactive questions. The question involve setting a folder location, making sure folder location is right etc for which answers might be default or custom.
I looked into the expect module of ansible but seems like it does not solve my purpose.
- expect:
command: passwd username
responses:
(?i)password: "MySekretPa$$word"
I don't need the command but it's required. Instead I am looking for something that could regex Are you sure you want to continue [y|n]? [n]: for which I want to send the default out By sending return or typing n as a response and for example Backup directory [/tmp] for which the response would be Carriage return.
I don't need the command but it's required. Instead I am looking for something that could regex Are you sure you want to continue [y|n]? [n]:
The module requires a command because you have to run something to get any output.
You obviously do have a command in mind, because you've run it manually and seen the output it produces. That's what you should be plugging into the module.
Alternatively, you can write a pexpect script yourself and use the command or shell modules to run it.
I've figured out a way that works for me. I piped in the arguments to the shell script which when run manually needs the answers. Like ./shell.sh <<< 'answer1\nanswer2\n' which works perfectly for me. This I have added to the task.
I have a cron job to run a sed shell called sedcmd.sh to pre-process some json data. When i am in the proper directory I manually run it with
. ./sedcmd.sh
And it works. The shell itself works fine.
for reference one of the commands inside looks like
sed -i '/^\s*$/d' /home/school/Desktop/Programs/rawjsondata.txt
my cronjob looks like
5 * * * * . ./home/school/Desktop/Programs/sedcmd.sh
I get the error "No such file or directory found". What am I doing wrong. I've triple checked for any random spelling errors. I also can't seem to run sedcmd.sh from any other directory even when i give the entire file path, so its definitely something I'm doing wrong. My thoughts for solutions are either
I should add sedcmd.sh to my $PATH or bashrc so i can call it from anywhere. Which I don't know how to do.
OR
figure out how to call it correctly from crontab. Which I also dont know how to do.
When you are running a script from terminal, you do:
./script_name.sh but to execute the same script from crontab you do something like 5 * * * * /path/to/script/script_name.sh
As Sam has got this answer from the comments,posting the answer as community wiki.
I have a ruby script containing system command like http://gist.github.com/235833, while I ran this script from shell, it works correctly, but when I added it to my cron job list, it doesn't work any more, the cron job is like:
10/* * * * * cd /home/hekin; /usr/bin/ruby my_script.rb
any idea what's going wrong with what i've done? Thank you.
Thank you all for your answers.
It's my mistake.
Since I'm using ssh key forwarding on the local machine, while I executed the script from the shell, the ssh key forwarding related environment variables are all sitting there, but from cron job context, those environment variables are missing.
Try to separate the things that might go wrong. The ones I can think of are:
The cron syntax - is the time value given legal and fitting your shell?
Permissions - execute permissions and read permissions for the relevant directory and file
Quoting - what scope does cron cover? Does it run only the first command?
In order to dissect this, I suggest you first run a really simple cron job, like 'ls'. Next run a single-liner script. Next embed your commands in a shell-script file. Somewhere along these lines you should find the problem.
The problem is your environment. While testing in your shell its fully equipped and boosted by your shell environment. While running under cron its very, very stripped down.
Where is the destination "." for your script? I guess it will be "/" and may not "$HOME" thus your script won't be able to write at that location and fails. Try using an absolut path for the destination.
This keeps happening to me all the time:
1) I write a script(ruby, shell, etc).
2) run it, it works.
3) put it in crontab so it runs in a few minutes so I know it runs from there.
4) It doesnt, no error trace, back to step 2 or 3 a 1000 times.
When I ruby script fails in crontab, I can't really know why it fails cause when I pipe output like this:
ruby script.rb >& /path/to/output
I sorta get the output of the script, but I don't get any of the errors from it and I don't get the errors coming from bash (like if ruby is not found or file isn't there)
I have no idea what environmental variables are set and whether or not it's a problem. Turns out that to run a ruby script from crontab you have to export a ton of environment variables.
Is there a way for me to just have crontab run a script as if I ran it myself from my terminal?
When debugging, I have to reset the timer and go back to waiting. Very time consuming.
How to test things in crontab better or avoid these problems?
"Is there a way for me to just have crontab run a script as if I ran it myself from my terminal?"
Yes:
bash -li -c /path/to/script
From the man page:
[vindaloo:pgl]:~/p/test $ man bash | grep -A2 -m1 -- -i
-i If the -i option is present, the shell is interactive.
-l Make bash act as if it had been invoked as a login shell (see
INVOCATION below).
G'day,
One of the basic problems with cron is that you get a minimal environment being set by cron. In fact, you only get four env. var's set and they are:
SHELL - set to /bin/sh
LOGNAME - set to your userid as found in /etc/passwd
HOME - set to your home dir. as found in /etc/passwd
PATH - set to "/usr/bin:/bin"
That's it.
However, what you can do is take a snapshot of the environment you want and save that to a file.
Now make your cronjob source a trivial shell script that sources this env. file and then executes your Ruby script.
BTW Having a wrapper source a common env. file is an excellent way to enforce a consistent environment for multiple cronjobs. This also enforces the DRY principle because it gives you just one point to update things as required, instead of having to search through a bunch of scripts and search for a specific string if, say, a logging location is changed or a different utility is now being used, e.g. gnutar instead of vanilla tar.
Actually, this technique is used very successfully with The Build Monkey which is used to implement Continuous Integration for a major software project that is common to several major world airlines. 3,500kSLOC being checked out and built several times a day and over 8,000 regression tests run once a day.
HTH
'Avahappy,
Run a 'set' command from inside of the ruby script, fire it from crontab, and you'll see exactly what's set and what's not.
To find out the environment in which cron runs jobs, add this cron job:
{ echo "\nenv\n" && env|sort ; echo "\nset\n" && set; } | /usr/bin/mailx -s 'my env' you#example.com
Or send the output to a file instead of email.
You could write a wrapper script, called for example rbcron, which looks something like:
#!/bin/bash
RUBY=ruby
export VAR1=foo
export VAR2=bar
export VAR3=baz
$RUBY "$*" 2>&1
This will redirect standard error from ruby to the standard output. Then you run rbcron in your cron job, and the standard output contains out+err of ruby, but also the "bash" errors existing from rbcron itself. In your cron entry, redirect 2>&1 > /path/to/output to get output+error messages to go to /path/to/output.
If you really want to run it as yourself, you may want to invoke ruby from a shell script that sources your .profile/.bashrc etc. That way it'll pull in your environment.
However, the downside is that it's not isolated from your environment, and if you change that, you may find your cron jobs suddenly stop working.