Set-up
I have 3 .txt files containing commands to be executed each day.
The berlin_run.txt file executes a.o. the 2 other .txt files. The file is the following,
#!/bin/bash
cd /path/to/folder/containing/berlin_run.txt
PATH=$PATH:/usr/local/bin
export PATH
./spider_apartments_run.txt
./spider_rooms_run.txt
python berlin_apartments_ads.py;python berlin_rooms.py
When I cd to /path/to/folder/containing/berlin_run.txt in my MacOS Terminal, and execute the ./berlin_run.txt command, everything works fine.
It is my understanding that ./ opens the berlin_run.txt, and that #!/bin/bash ensures that the subsequent lines in the berlin_run.txt are automatically executed upon opening.
Problem
I want to automate the execution of berlin_run.txt.
I have written the following cronjob,
10 13 * * * /path/to/folder/containing/berlin_run.txt
It is my understanding that this cronjob should open the berlin_run.txt each day at 13:10. Assuming that is correct, #!/bin/bash should execute all the subsequent lines. But nothing seems to happen.
Where and what am I doing wrong here?
Related
I have a bash script that I can execute with cd ~/Documents/Code/shhh/ && ./testy if i'm in any directory on my computer and that successfully pushes to github which is what i want.
I'm trying to schedule a cron job to do this daily so I ran crontab -e which opens a nano editor and then I put 30 20 * * * cd ~/Documents/Code/shhh/ && ./testy to run daily at 10:30pm and hit control O, enter and control X. But still it it didn't execute. When I type crontab -l it shows my command & I have a You have new mail. message when I open a new window. Still my command doesn't execute even though it will when I run it from any other directory.
I think my crontab job is at /var/at/tmp so I ran 30 20 * * * cd ../../../Users/squirrel/Documents/Code/shhh/ && ./testy but still nothing, even though it does work when I write it out myself from that directory. Sidenote, I can't enter into the tmp folder even after using sudo
OK When I type in mail I see a lot message and inside i get this error
---------------Checking Status of 2---------------
[master 0c1fff8] hardyharhar
1 file changed, 1 insertion(+), 1 deletion(-)
fatal: could not read Username for 'https://github.com': Device not configured
When you by default open nano Input_file it opens it up in INSERT mode(unlike vi where we have to explicitly go to INSERT mode by pressing i key). Now when you have done CTNRL+O it will then ask you if you want to save changes to opened file or not eg--> File Name to Write: Input_file If you press ENTER then it will save it and will be back on the screen(to your Input_file) where you entered new line. Now you could press CONTRL+X to come out of Input_file. May be you are stuck after saving it and want to come out then try this out once?
crontab -e does not edit the crontab file "live" at all -- changes are saved to the active file only after you save changes and exit the editor.
It also sounds like you may be using incorrect directory paths. The command in a crontab entry will generally be executed starting from the user's home directory. So if your home directory is /Users/squirrl, the command cd ../parent_directory/ will try to move to /Users/parent_directory. I suspect this is not what you want.
Finally, note that cron jobs run with a very minimal environment, without running most of your usual shell setup files (e.g. .bashrc). Most notably, if your script uses any commands that aren't in /bin or /usr/bin, you'll either need to use explicit full paths to them, or change the PATH variable to include the directories they're in.
Below is my .sh file
sh summaryByClient.sh $1 - takes around 10 mins to fetch the required data
mv summary.html ~/public_html/chats/ - **this is not happening**
exit 0
I do not understand why mv summary.html ~/public_html/chats/ this is not working inside .sh file, However I am able to mv separately using the same above command.
Could you be running into the issue from this answer with expanding the user's home directory? What happens if you write your script like this:
#!/bin/bash
# Other tasks to retrieve summary.html done here
mv ./summary.html $HOME/public_html/chats/
exit 0
Also, it's always a good idea to check whether the destination directory exists before a mv command. Examples are shown in this answer
I have hadoop installed on centOS system. I have a shell script which merges all the small files of HDFS generated at some particular hour folder location into one single file at another location at hdfs.
The shell works perfectly OK when invoked.
I then placed the shell to run as a cron job at 01:30 AM everyday.
I typed crontab -e and pasted this:
30 1 * * * /home/hadoop/tmp/cron-merge-files.sh > /home/hadoop/tmp/cron-merge-files.txt
But the merge operation does not happen. I see at /var/log/cron file that at 01:30 AM this entry comes but I cant see those files merged at hdfs. When I simply execute
the shell script, then it works perfectly OK and does the said operation written inside the script.
Jul 8 01:30:01 ip-10-1-3-111 CROND[2265463]: (hadoopuser) CMD (/home/hadoop/tmp/cron-merge-files.sh > /home/hadoop/tmp/cron-merge-files.txt)
The content of /home/hadoop/tmp/cron-merge-files.txt is a single echo statement which is written inside a loop. The loop is supposed to run 24 times, and it prints it
24 times.
I am not sure what is happening.
I got the solution for this problem from another forum. There was some problem with the environment variables not getting picked up when same script was run through crontab. Just exported .bash_profile to my script and it worked.
I just during the weekend decided to try out zsh and have a bit of fun with it. Unfortunately I'm an incredible newbie to shell scripting in general.
I have this folder with a file, which filename is a hash (4667e85581f80b6936f8811f0a7493c70eae4ee7) without a file-extension.
What I would like to do is copy this file to another folder and rename it to "screensaver.png".
I've tried with the following code:
#!/usr/bin/zsh
KUVVA_CACHE="$HOME/Library/Containers/com.kuvva.Kuvva-Wallpapers/Data/Library/Application Support/Kuvva"
DEST_FOLDER="/Library/Desktop Pictures/Kuvva/$USERNAME/screensaver.png"
for wallpaper in ${KUVVA_CACHE}; do
cp -f ${wallpaper} ${DEST_FOLDER}
done
This returns the following error:
cp: /Users/Morten/Library/Containers/com.kuvva.Kuvva-Wallpapers/Data/Library/Application Support/Kuvva is a directory (not copied).
And when I try to echo the $wallpaper variable instead of doing "cp" then it just echo's the folder path.
The name of the file changes every 6 hour, which is why I'm doing the for-loop. So I never know what the name of the file will be, but I know that there's always only ONE file in the folder.
Any ideas how I can manage to do this? :)
Thanks a lot!
Morten
It should work with regular filename expansion (globbing).
KUVVA_CACHE="$HOME/Library/Containers/com.kuvva.Kuvva-Wallpapers/Data/Library/Application Support/Kuvva/"
And then copy
cp -f ${KUVVA_CACHE}/* ${DEST_FOLDER}
You can add the script to your crontab so it will be run at a certain interval. Edit it using 'crontab -e' and add
30 */3 * * * /location/of/your/script
This will run it every third hour. First digit is minutes. Star indicates any. Exit the editor by pressing the escape-key, then shift+: and type wq and press enter. These vi-commands.
Don't forget to 'chmod 0755 file-name' the script so it becomes executable.
Here is the script.
#!/bin/zsh
KUVVA_CACHE="$HOME/Library/Containers/com.kuvva.Kuvva-Wallpapers/Data/Library/Application Support/Kuvva"
DEST_FOLDER="/Library/Desktop Pictures/Kuvva/$USERNAME/screensaver.png"
cp "${KUVVA_CACHE}/"* "${DEST_FOLDER}"
I've written a bash script that executes a python script to write a file to a directory, then sends that file to Amazon S3. When I execute the script from the command line it executes perfectly, but when I run it with cron, the file writes to the directory, but never gets sent to S3. I must be doing something wrong with cron.
Here is the bash script:
#!/bin/bash
#python script that exports file to home directory
python some_script.py
#export file created by python to S3
s3cmd put /home/bitnami/myfile.csv s3://location/to/put/file/myfile.csv
Like I said before, manually executing works fine using ./bash_script.sh. When I set up the cron job, the file writes to the directory, but never gets sent to S3.
my cron job is:
18 * * * * /home/bitnami/bash_script.sh
Am I using cron incorrectly? Please help.
Cron looks OK, however your path to the .py file will not be found.
You will have to add a path or home like:
location=/home/bitnami/
python $location/some_script.py
Also s3cmd needs to be located correctly:
/bin/s3cmd
Alternative might also need to load your user environment first before executing the script to find username/password/ssh key for s3cmd