I'm using ack as part of a bash script to build a list of MP4 files from a mysqldump. The dump file is about 15mb.
Here's my line:
ack -o "https??://cdn.host.com\S+?\.mp4" /home/me/.dump.sql > /home/me/.mp4-matches.txt
It works fine when running bash script by hand. We get this in .mp4-matches.txt:
https://cdn.host.com/url/t_Foo/Foo.mp4
https://cdn.host.com/url/t_Bar/Bar.mp4
But when running the very same command by itself as a cronjob, it produces an empty file.
I can't figure out why it's not working in cron.
I've tried fiddling with PATH, SHELL, etc in the crontab to try ensure environment is the same as running it by hand. Nothing made a difference.
I've tried using all hard paths in crontab to ack /usr/bin/ack just to be sure. Didn't make a difference.
I've tried using bash -l to start the script. Didn't make a difference.
What am I doing wrong?
Edits for further info:
I am running Debian 7 but I also tested on Ubuntu 18.04 and same issue appears there too.
Server is running exim4, all working. Nothing is being sent to the MAILTO address about this line in the cron.
There's nothing in /var/log/syslog or /var/log/messages concerning errors from cron
Using bash #!/bin/bash
There are no percent signs (%) anywhere in the crontab.
Crontab looks like this:
MAILTO=myemail#address.com
BASH=/bin/bash
SHELL=/bin/bash
PATH=/usr/local/bin:/usr/bin:/bin
27 9 * * * /usr/bin/ack -o "https??://cdn.host.com\S+?\.mp4" /home/me/.dump.sql > /home/me/.mp4-matches.txt
Yes, it's my crontab, not root.
There is no .ackrc file being read. It does not exist anywhere on my system. find / -iname ".ackrc" returned nothing.
A simple expression "https://cdn.host.com/url/t_Foo/Foo.mp4" still returns an empty file in cron.
Both single quotes and double quotes produce empty file in cron.
STERR seems to provide nothing also. Running /usr/bin/ack -o 'https://cdn.host.com/url/t_Foo/Foo.mp4' /home/me/.dump.sql > /home/me/.mp4-matches.txt 2> /home/me/.ackerror.txt returned both .mp4-matches.txt and .ackerror.txt files as 0 bytes.
This should work:
* * * * * ack -o "https??://cdn.host.com\S+?\.mp4" /home/me/.dump.sql </dev/null > /home/me/.mp4-matches.txt
Also you probably can use your line with the the --nofilter option
Related
I read the other related topics but they didn't help me.
I have a shell script which checks if my python script is not running,it will run it. Otherwise it will just skip and do nothing.
It totally works when I use:
bash myshellscrip.sh
And I get the result that I want which is doing some tasks and sending emails to some correspondents. However, when I try to run this particular shell script on crontab, it doesn't send out the emails and doesn't do the other tasks.
I tried the following on crontab and none of them worked.
* * * * * /bin/bash /path/to/my/script/myshellscrip.sh
* * * * * /bin/bash /path/to/my/script/myshellscrip.sh >> /some/other/path/output.txt
When I save the changes into 'output.txt' file, it creates the file but it doesn't send the emails or doing other tasks.
I also tried the option of reboot because I need this program to run at start up too, and this didn't work:
#reboot /bin/bash /path/to/my/script/myshellscrip.sh
Does anyone know how to fix it?
EDIT:
As I was checking with the simplest shell scrip like:
#!/bin/sh
/usr/bin/python /home/pi/DCA/code.py
My crontab wouldn't have any output in my output.txt file although my code.py have something printing out, too.
However, when I use a very simple python code for example only a 'print' statement it will run and save the output into output.txt.
Seems like your shell script crashes / stops before it can do something (possibly due to the environment being different or permission issues). You can check /var/log/syslog to find out.
You could try removing /bin/bash, I don't think that's necessary?
Run the cron job in debug mode. for that, Add -x to the bash command on the cronjob and save their output in the file.
bash -x /path/to/script.sh >> /path/to/the/output.txt
You can find the problem.
Apparently crontab was running my script several times. So I tried to use different locking mechanisms to put a lock around my scrip but only using flock worked for me. In my crontab I added this line:
* * * * * /usr/bin/flock -n /tmp/ms.lockfile /bin/bash /path/to/my/script/myShellScript.sh
I have the following entry in my crontab:
0,30 7-18 * * 1-5 cd /path/to/scrapers && scrapy crawl funny_quotes &>> $(date "+/home/foobar/logs/\%Y\%m\%d.funny.log"
This entry is supposed to run every half hour, on weekdays and append the output to the log file each time it's run. I have tested the syntax online, using this handy tool, and the syntax is correct.
However, the job doesn't get run. What's worse, the log file is created (but has no contents - file size 0), so I have no diagnostic information to go by.
The command cd /path/to/scrapers && scrapy crawl funny_quotes runs perfectly when I type it at the command, and there is copious amounts of information output to the console, from scrapy.
Why does the cronjob fail to run sccessfully - and why is nothing being piped to the log file?
Check your cron logs
grep CRON /var/log/syslog
I am sure you are getting error something like scrapy - command not found or something similar.
To fix it, do this
Enter and copy the output of echo $PATH from shell.
And then open crontab -e
At the very top of file, write PATH=YOUR_COPIED_CONTENTS
And that should work.
Following is the entry in the crontab:
MAILTO=abc#gmail.com
45 14 * * * /home/user/simple.sh
I've also done chmod +x on the simple.sh But the crontab does not run, it doesn't even send an email.
pgrep cron shows an Id. I also tried bouncing crond. But no luck!
Could someone please point out the mistake here
The simple.sh script is:
#! /bin/bash
echo hello
Thanks
Since you are doing a echo within the cron job script, you need to capture its output somewhere.
Your shebang and file mode (using chmod +x) are all right, so those aren't the issue here and running without /bin/sh should work fine.
Try using the following to see the output in cron.log file (This runs every minute)
* * * * * /home/user/simple.sh >> /home/user/cron.log
Note that cron jobs run in separate subprocess shell, with reduced environment, so its output won't be visible on your terminal.
Regarding sending of email - you need to have some mail package (like postman, mutt etc) configured for the cron daemon to send out error mails.
Do not use relative paths, but absolute ones. Also, indicate the binary running the script, that is /bin/sh (or whatever coming from which sh):
45 14 * * * /bin/sh /path/to/script/simple.sh
Maybe there shouldn't be a space in line 1 of your .sh script:
#! /bin/bash
to
#!/bin/bash
Although I could see why it would still seem to work from when invoked in an interactive shell (# could merely comment out the rest of the line).
Still, I'd guess at worst it'd merely ignore that line and inherit cron's interpreter of /bin/sh
I can't seem to run a simple bash file as a cron job that runs once a minute.
#!/bin/bash
NET_INTERFACE=eth0
CURRENT_IP=`/sbin/ifconfig $NET_INTERFACE | sed -n "/inet addr:.*255.255.25[0-5].[0-9]/{s/.*inet addr://; s/ .*//; p}"`
wget -q --delete-after "http://abc.co.uk/raspiUpdate.php?pi=1&ip=${CURRENT_IP}"
In the crontab I have:
* * * * * /home/user/Scripts/script.sh
Where am I going wrong?
Thanks
To troubleshoot this cron issue a few steps come to mind:
double-check the scripts file name you put in your crontab:
$ ls -lah /home/user/Scripts/script.sh
$ cat /home/user/Scripts/script.sh
check that that script is executable by the user the crontab is installed for. i.e. if you use /etc/crontab, it should be executable by root, if you used
$ crontab -e
it should be executable by your userid.
Actually running that script should already give you a hint.
look for hints provided by cron. Depending on what distribution you run, the file cron logs to is different. For debian it should be /var/log/syslog IIRC
If you can't find an error message anywhere, it's hard to tell what's wrong. Generally if I reach that point, I stop and go to sleep or do something else.
[edit]
And when I return, i often find the error. :)
p.s.:
i recently had problems with dots in the filename. Some cron implementations ignore files in /etc/cron.* with dots in their filename. Documented here: http://manpages.ubuntu.com/manpages/xenial/en/man8/cron.8.html as Debian specific. But I think I had this issue also on Fedora.
I have a bash script mysql_cron.sh that runs mysqldump
#!/bin/bash
/usr/local/mysql/bin/mysqldump -ujoe -ppassword > /tmp/somefile
This works fine. I then call it from cron:
20 * * * * /home/joe/mysql_cron.sh
and this creates the file /tmp/somefile, but the file is always empty. I have tried adding a
source /home/joe/.bash_profile
to the script to make sure cron has the right env variables, but that doesn't help. I see many other people having this problem but have found no solution. I've also tried the '>' operator in the crontab to cat any cron errors to a file, but that doesn't seem to generate any errors. Any troubleshooting ideas welcomed. Thanks!
Add output of error information to file (as Damp has said), so that you can check if there is any error:
#!/bin/bash
/usr/local/mysql/bin/mysqldump -ujoe -ppassword > /tmp/somefile 2>&1
You can also take a look at MySQL's log files at /var/log in case there is some hint there.
Add this line to your script and compare the result between running it from cron versus running it directly:
env > /tmp/env.$$.out
The $$ will be replaced in the resulting filename by the PID of the parent process (cron or the shell). You should be able to diff the two files and see if anything significant is different between the two environments.