I have a bashscript that I'm running with crontab. Unfortunately, a script that works fine when run manually fails with the error:
Syntax error: "(" unexpected (expecting "}")
Where the line in question is line 22 which is:
declare -a PREV_TOTAL=( $(for i in ${range[#]}; do echo 0; done) )
In the larger context:
TOTAL_CPU_USAGE=0
TOTAL_CPU=$(grep -c ^processor /proc/cpuinfo) #set number of CPUs to check for
declare -a 'range=({'"0..$TOTAL_CPU"'})'
let "TOTAL_CPU=$TOTAL_CPU - 1"
#declare array of size TOTAL_CPU to store values (eg. 8 cpus makes arrays of size 8)
declare -a PREV_TOTAL=( $(for i in ${range[#]}; do echo 0; done) )
declare -a PREV_IDLE=( $(for i in ${range[#]}; do echo 0; done) )
This works when manually just fine, but I don't understand what I'm doing wrong that causes cron to give this error? If you know I'd be very appreciative. Thanks.
EDIT: My crontab looks like this:
# m h dom mon dow command
SHELL=/bin/bash
#reboot cd /home/ubuntu/waste-cloud-computing/probe && probe.sh >> /var/log/somelogfile.log 2>&1
And I access it with sudo crontab -e. I'm still getting the issue while providing the SHELL variable.
EDIT 1: Thanks to some help I got past the syntax issues by ensuring the shell was using bash. Now I get the error, /bin/bash: probe.bash: command not found. I assume its some kind of PATH issue, but which bash returns /bin/bash so it seems normal to me. Maybe someone knows what's up?
cron jobs are run by sh by default, not bash. If you are using ubuntu/vixiecron, you can set the SHELL env variable at the top of the crontab to make cron run the commands in your crontab with bash.
SHELL=/bin/bash
If the script you want to be run is a bash script, make sure you have a shebang at the first line:
#!/bin/bash
Also note that there will be other potential troubleshooting steps if your scripts depend on a particular user's profile, env vars, etc. depending on which crontab you are editing.
Thanks to the help of the people here I found my issue was not syntax but rather the use of sh over bash. This was fixed by setting the crontab this way so future users can see:
# m h dom mon dow command
SHELL=/bin/bash
#reboot cd /home/ubuntu/waste-cloud-computing/probe && ./probe.sh >> /var/log/somelogfile.log 2>&1
The key points are the SHELL variable being set and the ./ before running the script.
Related
I have a Bash script, and in it are calls for two Python scripts. While using crontab the Bash script runs just fine but the Python scripts don't do anything, as if they aren't called or interpreted. I know it could all be done in Python, but I wanted to learn some Bash so I decided to mix them.
What might the issue be? It's hard to find answers for this as no one would mix Bash and Python since it's definitely not something useful.
My crontab is this:
10 * * * * cd ~/directory && /bin/bash ~/directory/script.sh >> output.txt
I can see Python is not doing anything by taking a look at the log.txt file that I create with Bash (code below)
The Bash script looks something like this:
#! /bin/bash
...
var=$(python3 app.py)
...
now=$(date + ...)
echo $now $var >> log.txt
The code works fine in the terminal, I also have a venv and environmental variables so I don't know how impactful that could be.
(I put the output.txt in crontab just to debug, if the program was working I would only use the log.txt in the script).
(Posted the solution on behalf of the question author to move it to the answer space).
The crontab has no access to the environmental variables that I had declared in .bashrc. Declaring the variables in crontab -e worked in my OS Fedora Linux 35.
ENV_VAR="a"
ENV_VAR2="b"
10 * * * * cd ~/directory && /bin/bash ~/directory/script.sh >> output.txt
Might work if you have env variables in your Python script and run into the same problem using Fedora 35.
I have written the following code:
#!/bin/bash
#Simple array
array=(1 2 3 4 5)
echo ${array[*]}
And I am getting error:
array.sh: 3: array.sh: Syntax error: "(" unexpected
From what I came to know from Google, that this might be due to the fact that Ubuntu is now not taking "#!/bin/bash" by default... but then again I added the line but the error is still coming.
Also I have tried by executing bash array.sh but no luck! It prints blank.
My Ubuntu version is: Ubuntu 14.04
Given that script:
#!/bin/bash
#Simple array
array=(1 2 3 4 5)
echo ${array[*]}
and assuming:
It's in a file in your current directory named array.sh;
You've done chmod +x array.sh;
You have a sufficiently new version of bash installed in /bin/bash (you report that you have 4.3.8, which is certainly new enough); and
You execute it correctly
then that should work without any problem.
If you execute the script by typing
./array.sh
the system will pay attention to the #!/bin/bash line and execute the script using /bin/bash.
If you execute it by typing something like:
sh ./array.sh
then it will execute it using /bin/sh. On Ubuntu, /bin/sh is typically a symbolic link to /bin/dash, a Bourne-like shell that doesn't support arrays. That will give you exactly the error message that you report.
The shell used to execute a script is not affected by which shell you're currently using or by which shell is configured as your login shell in /etc/passwd or equivalent (unless you use the source or . command).
In your own answer, you say you fixed the problem by using chsh to change your default login shell to /bin/bash. That by itself should not have any effect. (And /bin/bash is the default login shell on Ubuntu anyway; had you changed it to something else previously?)
What must have happened is that you changed the command you use from sh ./array.sh to ./array.sh without realizing it.
Try running sh ./array.sh and see if you get the same error.
Instead of using sh to run the script,
try the following command:
bash ./array.sh
I solved the problem miraculously. In order to solve the issue, I found a link where it was described to be gone by using the following code. After executing them, the issue got resolved.
chsh -s /bin/bash adhikarisubir
grep ^adhikarisubir /etc/passwd
FYI, "adhikarisubir" is my username.
After executing these commands, bash array.sh produced the desired result.
When I type the following in a terminal ./DHT 11 4 it works and saves all data to mysql correctly.
id (1), temp (29), hum (37), date (2015...)
When I add it to a crontab it does not work correctly.
id (1), temp (0 or empty), hum (0 or empty), date (2015...)
sh script:
#!/bin/bash
#DHT11
SCRIPT="/var/www/ErnestynoFailai/scripts/DHT 11 4"
#DHT22
#SCRIPT="/root/to/folder/DHT 22 4"
#AM2302
#SCRIPT="/root/to/folder/DHT 2302 4"
TEMP_AND_HUM=""
while [[ $TEMP_AND_HUM == "" ]]
do
TEMP_AND_HUM=`$SCRIPT | grep "Temp"`
done
TEMP=`echo "$TEMP_AND_HUM" | cut -c8-9`
HUM=`echo "$TEMP_AND_HUM" | cut -c21-22`
myqsl_user="root"
myqsl_pw="pw"
myqsl_database="DHT"
today=`date +"%Y-%m-%d %T"`
query="INSERT INTO DHT11 (temp, hum, date) VALUES ('$TEMP', '$HUM', '$today');"
mysql --user=$myqsl_user --password=$myqsl_pw $myqsl_database << EOF
$query
EOF
And crontab:
*/1 * * * * /var/www/ErnestynoFailai/scripts/write_DHT11_to_db.sh
What can be wrong?
Long time ago, it happened on some systems that cron didn't start shell scripts, only binaries. So you had to indicate explicitely which interpreter to use in the crontab line
*/1 * * * * /bin/bash /var/www/ErnestynoFailai/scripts/write_DHT11_to_db.sh
I didn't check since, and I dont know what system you are using. On debian/jessie, it is told in the crontab 5 manpage that the command is executed by /bin/sh, or the shell specified by the SHELL variable in the crontab file.
See https://superuser.com/questions/81262/how-to-execute-shell-script-via-crontab
Probably you have a problem of different environment settings. For debugging, an easy way is to include a line like the following to the beginning of your script:
set >/tmp/envlog.txt
Then compare its contents once created when you run your script directly and once using crontab.
Another way for debugging is:
exec >/tmp/scriptoutput.txt 2>&1
set -x
With this commands, the full output of your script will be redirected to the specified file.
Most often, the PATH variable is wrong. Instead of
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
You often only have a reduces version of it:
/usr/sbin:/usr/bin:/sbin:/bin
This means that some commands cannot be found. If you find a command which doesn't work, try finding out where it is located using:
$ which mysql
/usr/bin/mysql
I have two shell scripts, a.sh and b.sh.
How can I call b.sh from within the shell script a.sh?
There are a couple of different ways you can do this:
Make the other script executable with chmod a+x /path/to/file(Nathan Lilienthal's comment), add the #!/bin/bash line (called shebang) at the top, and the path where the file is to the $PATH environment variable. Then you can call it as a normal command;
Or call it with the source command (which is an alias for .), like this:
source /path/to/script
Or use the bash command to execute it, like:
/bin/bash /path/to/script
The first and third approaches execute the script as another process, so variables and functions in the other script will not be accessible.
The second approach executes the script in the first script's process, and pulls in variables and functions from the other script (so they are usable from the calling script).
In the second method, if you are using exit in second script, it will exit the first script as well. Which will not happen in first and third methods.
Check this out.
#!/bin/bash
echo "This script is about to run another script."
sh ./script.sh
echo "This script has just run another script."
There are a couple of ways you can do this. Terminal to execute the script:
#!/bin/bash
SCRIPT_PATH="/path/to/script.sh"
# Here you execute your script
"$SCRIPT_PATH"
# or
. "$SCRIPT_PATH"
# or
source "$SCRIPT_PATH"
# or
bash "$SCRIPT_PATH"
# or
eval '"$SCRIPT_PATH"'
# or
OUTPUT=$("$SCRIPT_PATH")
echo $OUTPUT
# or
OUTPUT=`"$SCRIPT_PATH"`
echo $OUTPUT
# or
("$SCRIPT_PATH")
# or
(exec "$SCRIPT_PATH")
All this is correct for the path with spaces!!!
The answer which I was looking for:
( exec "path/to/script" )
As mentioned, exec replaces the shell without creating a new process. However, we can put it in a subshell, which is done using the parantheses.
EDIT:
Actually ( "path/to/script" ) is enough.
If you have another file in same directory, you can either do:
bash another_script.sh
or
source another_script.sh
or
. another_script.sh
When you use bash instead of source, the script cannot alter environment of the parent script. The . command is POSIX standard while source command is a more readable bash synonym for . (I prefer source over .). If your script resides elsewhere just provide path to that script. Both relative as well as full path should work.
Depends on.
Briefly...
If you want load variables on current console and execute you may use source myshellfile.sh on your code. Example:
#!/bin/bash
set -x
echo "This is an example of run another INTO this session."
source my_lib_of_variables_and_functions.sh
echo "The function internal_function() is defined into my lib."
returned_value=internal_function()
echo $this_is_an_internal_variable
set +x
If you just want to execute a file and the only thing intersting for you is the result, you can do:
#!/bin/bash
set -x
./executing_only.sh
bash i_can_execute_this_way_too.sh
bash or_this_way.sh
set +x
You can use /bin/sh to call or execute another script (via your actual script):
# cat showdate.sh
#!/bin/bash
echo "Date is: `date`"
# cat mainscript.sh
#!/bin/bash
echo "You are login as: `whoami`"
echo "`/bin/sh ./showdate.sh`" # exact path for the script file
The output would be:
# ./mainscript.sh
You are login as: root
Date is: Thu Oct 17 02:56:36 EDT 2013
First you have to include the file you call:
#!/bin/bash
. includes/included_file.sh
then you call your function like this:
#!/bin/bash
my_called_function
Simple source will help you.
For Ex.
#!/bin/bash
echo "My shell_1"
source my_script1.sh
echo "Back in shell_1"
Just add in a line whatever you would have typed in a terminal to execute the script!
e.g.:
#!bin/bash
./myscript.sh &
if the script to be executed is not in same directory, just use the complete path of the script.
e.g.:`/home/user/script-directory/./myscript.sh &
This was what worked for me, this is the content of the main sh script that executes the other one.
#!/bin/bash
source /path/to/other.sh
The top answer suggests adding #!/bin/bash line to the first line of the sub-script being called. But even if you add the shebang, it is much faster* to run a script in a sub-shell and capture the output:
$(source SCRIPT_NAME)
This works when you want to keep running the same interpreter (e.g. from bash to another bash script) and ensures that the shebang line of the sub-script is not executed.
For example:
#!/bin/bash
SUB_SCRIPT=$(mktemp)
echo "#!/bin/bash" > $SUB_SCRIPT
echo 'echo $1' >> $SUB_SCRIPT
chmod +x $SUB_SCRIPT
if [[ $1 == "--source" ]]; then
for X in $(seq 100); do
MODE=$(source $SUB_SCRIPT "source on")
done
else
for X in $(seq 100); do
MODE=$($SUB_SCRIPT "source off")
done
fi
echo $MODE
rm $SUB_SCRIPT
Output:
~ ❯❯❯ time ./test.sh
source off
./test.sh 0.15s user 0.16s system 87% cpu 0.360 total
~ ❯❯❯ time ./test.sh --source
source on
./test.sh --source 0.05s user 0.06s system 95% cpu 0.114 total
* For example when virus or security tools are running on a device it might take an extra 100ms to exec a new process.
pathToShell="/home/praveen/"
chmod a+x $pathToShell"myShell.sh"
sh $pathToShell"myShell.sh"
#!/bin/bash
# Here you define the absolute path of your script
scriptPath="/home/user/pathScript/"
# Name of your script
scriptName="myscript.sh"
# Here you execute your script
$scriptPath/$scriptName
# Result of script execution
result=$?
chmod a+x /path/to/file-to-be-executed
That was the only thing I needed. Once the script to be executed is made executable like this, you (at least in my case) don't need any other extra operation like sh or ./ while you are calling the script.
Thanks to the comment of #Nathan Lilienthal
Assume the new file is "/home/satya/app/app_specific_env" and the file contents are as follows
#!bin/bash
export FAV_NUMBER="2211"
Append this file reference to ~/.bashrc file
source /home/satya/app/app_specific_env
When ever you restart the machine or relogin, try echo $FAV_NUMBER in the terminal. It will output the value.
Just in case if you want to see the effect right away, source ~/.bashrc in the command line.
There are some problems to import functions from other file.
First: You needn't to do this file executable. Better not to do so!
just add
. file
to import all functions. And all of them will be as if they are defined in your file.
Second: You may be define the function with the same name. It will be overwritten. It's bad. You may declare like that
declare -f new_function_name=old_function_name
and only after that do import.
So you may call old function by new name.
Third: You may import only full list of functions defined in file.
If some not needed you may unset them. But if you rewrite your functions after unset they will be lost. But if you set reference to it as described above you may restore after unset with the same name.
Finally In common procedure of import is dangerous and not so simple. Be careful! You may write script to do this more easier and safe.
If you use only part of functions(not all) better split them in different files. Unfortunately this technique not made well in bash. In python for example and some other script languages it's easy and safe. Possible to make partial import only needed functions with its own names. We all want that in next bush versions will be done the same functionality. But now We must write many additional cod so as to do what you want.
Use backticks.
$ ./script-that-consumes-argument.sh `sh script-that-produces-argument.sh`
Then fetch the output of the producer script as an argument on the consumer script.
I normally have several problems with how cron executes scripts as they normally don't have my environment setup. Is there a way to invoke bash(?) in the same way cron does so I could test scripts before installing them?
Add this to your crontab (temporarily):
* * * * * env > ~/cronenv
After it runs, do this:
env - `cat ~/cronenv` /bin/sh
This assumes that your cron runs /bin/sh, which is the default regardless of the user's default shell.
Footnote: if env contains more advanced config, eg PS1=$(__git_ps1 " (%s)")$, it will error cryptically env: ": No such file or directory.
Cron provides only this environment by default :
HOME user's home directory
LOGNAME user's login
PATH=/usr/bin:/usr/sbin
SHELL=/usr/bin/sh
If you need more you can source a script where you define your environment before the scheduling table in the crontab.
Couple of approaches:
Export cron env and source it:
Add
* * * * * env > ~/cronenv
to your crontab, let it run once, turn it back off, then run
env - `cat ~/cronenv` /bin/sh
And you are now inside a sh session which has cron's environment
Bring your environment to cron
You could skip above exercise and just do a . ~/.profile in front of your cron job, e.g.
* * * * * . ~/.profile; your_command
Use screen
Above two solutions still fail in that they provide an environment connected to a running X session, with access to dbus etc. For example, on Ubuntu, nmcli (Network Manager) will work in above two approaches, but still fail in cron.
* * * * * /usr/bin/screen -dm
Add above line to cron, let it run once, turn it back off. Connect to your screen session (screen -r). If you are checking the screen session has been created (with ps) be aware that they are sometimes in capitals (e.g. ps | grep SCREEN)
Now even nmcli and similar will fail.
You can run:
env - your_command arguments
This will run your_command with empty environment.
Depending on the shell of the account
sudo su
env -i /bin/sh
or
sudo su
env -i /bin/bash --noprofile --norc
From http://matthew.mceachen.us/blog/howto-simulate-the-cron-environment-1018.html
Answering six years later: the environment mismatch problem is one of the problems solved by systemd "timers" as a cron replacement. Whether you run the systemd "service" from the CLI or via cron, it receives exactly the same environment, avoiding the environment mismatch problem.
The most common issue to cause cron jobs to fail when they pass manually is the restrictive default $PATH set by cron, which is this on Ubuntu 16.04:
"/usr/bin:/bin"
By contrast, the default $PATH set by systemd on Ubuntu 16.04 is:
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
So there's already a better chance that a systemd timer is going to find a binary without further hassle.
The downside with systemd timers, is there's a slightly more time to set them up. You first create a "service" file to define what you want to run and a "timer" file to define the schedule to run it on and finally "enable" the timer to activate it.
Create a cron job that runs env and redirects stdout to a file.
Use the file alongside "env -" to create the same environment as a cron job.
Don't forget that since cron's parent is init, it runs programs without a controlling terminal. You can simulate that with a tool like this:
http://libslack.org/daemon/
By default, cron executes its jobs using whatever your system's idea of sh is. This could be the actual Bourne shell or dash, ash, ksh or bash (or another one) symlinked to sh (and as a result running in POSIX mode).
The best thing to do is make sure your scripts have what they need and to assume nothing is provided for them. Therefore, you should use full directory specifications and set environment variables such as $PATH yourself.
The accepted answer does give a way to run a script with the environment cron would use. As others pointed out, this is not the only needed criteria for debugging cron jobs.
Indeed, cron also uses a non-interactive terminal, without an attached input, etc.
If that helps, I have written a script that enables painlessly running a command/script as it would be run by cron. Invoke it with your command/script as first argument and you're good.
This script is also hosted (and possibly updated) on Github.
#!/bin/bash
# Run as if it was called from cron, that is to say:
# * with a modified environment
# * with a specific shell, which may or may not be bash
# * without an attached input terminal
# * in a non-interactive shell
function usage(){
echo "$0 - Run a script or a command as it would be in a cron job, then display its output"
echo "Usage:"
echo " $0 [command | script]"
}
if [ "$1" == "-h" -o "$1" == "--help" ]; then
usage
exit 0
fi
if [ $(whoami) != "root" ]; then
echo "Only root is supported at the moment"
exit 1
fi
# This file should contain the cron environment.
cron_env="/root/cron-env"
if [ ! -f "$cron_env" ]; then
echo "Unable to find $cron_env"
echo "To generate it, run \"/usr/bin/env > /root/cron-env\" as a cron job"
exit 0
fi
# It will be a nightmare to expand "$#" inside a shell -c argument.
# Let's rather generate a string where we manually expand-and-quote the arguments
env_string="/usr/bin/env -i "
for envi in $(cat "$cron_env"); do
env_string="${env_string} $envi "
done
cmd_string=""
for arg in "$#"; do
cmd_string="${cmd_string} \"${arg}\" "
done
# Which shell should we use?
the_shell=$(grep -E "^SHELL=" /root/cron-env | sed 's/SHELL=//')
echo "Running with $the_shell the following command: $cmd_string"
# Let's route the output in a file
# and do not provide any input (so that the command is executed without an attached terminal)
so=$(mktemp "/tmp/fakecron.out.XXXX")
se=$(mktemp "/tmp/fakecron.err.XXXX")
"$the_shell" -c "$env_string $cmd_string" >"$so" 2>"$se" < /dev/null
echo -e "Done. Here is \033[1mstdout\033[0m:"
cat "$so"
echo -e "Done. Here is \033[1mstderr\033[0m:"
cat "$se"
rm "$so" "$se"
Another simple way I've found (but may be error prone, I'm still testing) is to source your user's profile files before your command.
Editing a /etc/cron.d/ script:
* * * * * user1 comand-that-needs-env-vars
Would turn into:
* * * * * user1 source ~/.bash_profile; source ~/.bashrc; comand-that-needs-env-vars
Dirty, but it got the job done for me. Is there a way to simulate a login? Just a command you could run? bash --login didn't work. It sounds like that would be the better way to go though.
EDIT: This seems to be a solid solution: http://www.epicserve.com/blog/2012/feb/7/my-notes-cron-directory-etccrond-ubuntu-1110/
* * * * * root su --session-command="comand-that-needs-env-vars" user1 -l
Answer https://stackoverflow.com/a/2546509/5593430 shows how to obtain the cron environment and use it for your script. But be aware that the environment can differ depending on the crontab file you use. I created three different cron entries to save the environment via env > log. These are the results on an Amazon Linux 4.4.35-33.55.amzn1.x86_64.
1. Global /etc/crontab with root user
MAILTO=root
SHELL=/bin/bash
USER=root
PATH=/sbin:/bin:/usr/sbin:/usr/bin
PWD=/
LANG=en_US.UTF-8
SHLVL=1
HOME=/
LOGNAME=root
_=/bin/env
2. User crontab of root (crontab -e)
SHELL=/bin/sh
USER=root
PATH=/usr/bin:/bin
PWD=/root
LANG=en_US.UTF-8
SHLVL=1
HOME=/root
LOGNAME=root
_=/usr/bin/env
3. Script in /etc/cron.hourly/
MAILTO=root
SHELL=/bin/bash
USER=root
PATH=/sbin:/bin:/usr/sbin:/usr/bin
_=/bin/env
PWD=/
LANG=en_US.UTF-8
SHLVL=3
HOME=/
LOGNAME=root
Most importantly PATH, PWD and HOME differ. Make sure to set these in your cron scripts to rely on a stable environment.
In my case, cron was executing my script using sh, which fail to execute some bash syntax.
In my script I added the env variable SHELL:
#!/bin/bash
SHELL=/bin/bash
I don't believe that there is; the only way I know to test a cron job is to set it up to run a minute or two in the future and then wait.