arithmetic syntax error in ksh if condition - shell

Part of my KornShell (ksh) script is as follows .
let "value=`awk NR==14 ${TEMP_DIR}/IR4723/count_part_UNVM.txt`"
let "var= value"
if [ $var -gt 0 ]
then
load_data ${count_part[$i]}
fi
first line in above part of script throw following error:
arithmetic syntax error LOADTODB/ShellsAndSQLs/IR4723/load_gen_tmp_tab.ksh[84]: test: 8
above script reading line number 14 from file count_part_UNVM.txt value 8.
I am stuck in this issue. Please help me to solve issue.

Is that a typo, or are you missing the '$' in front of value, i.e. (your 2nd line)
let "var=$value"
which I think should really be
let var="$value"
You can eliminate a lot of problems while you're building your script by turning on the shell debugging feature, i.e.
set -vx
let "var= value"
set +vx
Or put set -vx near the top of the script and see how all of the script is processed.
EDIT3 OR put -vx after #!/bin/bash on the first line, i.e. #!/bin/bash -vx.
EDIT2 I missed your first comment, yes, line1 looks problematic too. Try this
let value="$(awk 'NR==14' ${TEMP_DIR}/IR4723/count_part_UNVM.txt)"
Note that using backticks for cmd-substitution was declared as deprecated in the 'New Kornshell Language' ~ 1995. $( ) for cmd-substition is easily nestable and your best bet.
ALSO, note that an awk script (on a cmdline) should be presented as single argument, by surrounding it with single or dbl-quotes. (But I'm not absolutely certain you need quotes given that your 'program' has no spaces in it.) If the revision doesn't work, add print --"value=XX${value}XX after the assignment and edit your question above to show the output.
EDIT Per your comment/question, there are a few shell script debuggers around, but I never use them. Search for Rosenberg ksh book debugger, if you really want to try it.
The set -vx should really be called an "execution trace". It shows you the line or block of code before it is executed, then with any variables expanded to their values. That it the typical way people debug ksh scripts. You can also add statments like print -- "var=XX${var}XX" to see individual values of variables, but that can mess up your output and may required you to turn of those statments, forcing you to edit your script again.
I hope this helps.

Why are you using let in a ksh? Here is a suggestion for your code.
example.ksh
#!/bin/ksh
#Initialize Varables
TEMP_DIR=tempDir/IR4723
var=""
initialize(){
echo "Entering initialize"
echo "Exiting initialize"
}
function1(){
echo "Entering function1"
echo ${PWD}
file=${PWD}/${TEMP_DIR}/count_part_UNVM.txt
readLine "${file}"
echo "var: "${var}
if [[ ${var} -gt 0 ]]; then
load_data ${count_part[$i]}
fi
echo "Exiting function1"
}
createDir(){
echo "Entering createDir"
mkdir -p ${1}
echo "Exiting createDir"
}
createFileWithRandomCount(){
echo "Entering createFileWithRandomCount"
orgDirectory=${PWD}
cd ${1}
> ${2}
print ${RANDOM} >> ${2}
cd ${orgDirectory}
echo "Exiting createFileWithRandomCount"
}
readLine(){
echo "Entering readLine"
file=${1}
while read line
do
# display line or do somthing on $line
var="$line"
done <"$file"
echo "Exiting readLine"
}
load_data(){
echo "Entering load_data"
echo "Exiting load_data"
}
#-----------
#---Main----
#-----------
echo "Starting: "${0}" with Input Parameters: {1: "${1}" {2: "${2}" {3: "${3}
initialize #function call#
createDir ${TEMP_DIR} #function call#
createFileWithRandomCount ${TEMP_DIR} count_part_UNVM.txt #function call#
function1 #function call#
Output:
$ ksh -i example.ksh
Starting: example.ksh with Input Parameters: {1: {2: {3:
Entering initialize
Exiting initialize
Entering createDir
Exiting createDir
Entering createFileWithRandomCount
Exiting createFileWithRandomCount
Entering function1
/tmp
Entering readLine
Exiting readLine
var: 11984
Entering load_data
Exiting load_data
Exiting function1

Related

How to create a custom error message when anything in a bash scripts fails? [duplicate]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed last year.
The community reviewed whether to reopen this question last year and left it closed:
Original close reason(s) were not resolved
Improve this question
What is your favorite method to handle errors in Bash?
The best example of handling errors I have found on the web was written by William Shotts, Jr at http://www.linuxcommand.org.
He suggests using the following function for error handling in Bash:
#!/bin/bash
# A slicker error handling routine
# I put a variable in my scripts named PROGNAME which
# holds the name of the program being run. You can get this
# value from the first item on the command line ($0).
# Reference: This was copied from <http://www.linuxcommand.org/wss0150.php>
PROGNAME=$(basename $0)
function error_exit
{
# ----------------------------------------------------------------
# Function for exit due to fatal program error
# Accepts 1 argument:
# string containing descriptive error message
# ----------------------------------------------------------------
echo "${PROGNAME}: ${1:-"Unknown Error"}" 1>&2
exit 1
}
# Example call of the error_exit function. Note the inclusion
# of the LINENO environment variable. It contains the current
# line number.
echo "Example of error with line number and message"
error_exit "$LINENO: An error has occurred."
Do you have a better error handling routine that you use in Bash scripts?
Use a trap!
tempfiles=( )
cleanup() {
rm -f "${tempfiles[#]}"
}
trap cleanup 0
error() {
local parent_lineno="$1"
local message="$2"
local code="${3:-1}"
if [[ -n "$message" ]] ; then
echo "Error on or near line ${parent_lineno}: ${message}; exiting with status ${code}"
else
echo "Error on or near line ${parent_lineno}; exiting with status ${code}"
fi
exit "${code}"
}
trap 'error ${LINENO}' ERR
...then, whenever you create a temporary file:
temp_foo="$(mktemp -t foobar.XXXXXX)"
tempfiles+=( "$temp_foo" )
and $temp_foo will be deleted on exit, and the current line number will be printed. (set -e will likewise give you exit-on-error behavior, though it comes with serious caveats and weakens code's predictability and portability).
You can either let the trap call error for you (in which case it uses the default exit code of 1 and no message) or call it yourself and provide explicit values; for instance:
error ${LINENO} "the foobar failed" 2
will exit with status 2, and give an explicit message.
Alternatively shopt -s extdebug and give the first lines of the trap a little modification to trap all non-zero exit codes across the board (mind set -e non-error non-zero exit codes):
error() {
local last_exit_status="$?"
local parent_lineno="$1"
local message="${2:-(no message ($last_exit_status))}"
local code="${3:-$last_exit_status}"
# ... continue as above
}
trap 'error ${LINENO}' ERR
shopt -s extdebug
This then is also "compatible" with set -eu.
That's a fine solution. I just wanted to add
set -e
as a rudimentary error mechanism. It will immediately stop your script if a simple command fails. I think this should have been the default behavior: since such errors almost always signify something unexpected, it is not really 'sane' to keep executing the following commands.
Reading all the answers on this page inspired me a lot.
So, here's my hint:
file content: lib.trap.sh
lib_name='trap'
lib_version=20121026
stderr_log="/dev/shm/stderr.log"
#
# TO BE SOURCED ONLY ONCE:
#
###~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~##
if test "${g_libs[$lib_name]+_}"; then
return 0
else
if test ${#g_libs[#]} == 0; then
declare -A g_libs
fi
g_libs[$lib_name]=$lib_version
fi
#
# MAIN CODE:
#
###~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~##
set -o pipefail # trace ERR through pipes
set -o errtrace # trace ERR through 'time command' and other functions
set -o nounset ## set -u : exit the script if you try to use an uninitialised variable
set -o errexit ## set -e : exit the script if any statement returns a non-true return value
exec 2>"$stderr_log"
###~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~##
#
# FUNCTION: EXIT_HANDLER
#
###~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~##
function exit_handler ()
{
local error_code="$?"
test $error_code == 0 && return;
#
# LOCAL VARIABLES:
# ------------------------------------------------------------------
#
local i=0
local regex=''
local mem=''
local error_file=''
local error_lineno=''
local error_message='unknown'
local lineno=''
#
# PRINT THE HEADER:
# ------------------------------------------------------------------
#
# Color the output if it's an interactive terminal
test -t 1 && tput bold; tput setf 4 ## red bold
echo -e "\n(!) EXIT HANDLER:\n"
#
# GETTING LAST ERROR OCCURRED:
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #
#
# Read last file from the error log
# ------------------------------------------------------------------
#
if test -f "$stderr_log"
then
stderr=$( tail -n 1 "$stderr_log" )
rm "$stderr_log"
fi
#
# Managing the line to extract information:
# ------------------------------------------------------------------
#
if test -n "$stderr"
then
# Exploding stderr on :
mem="$IFS"
local shrunk_stderr=$( echo "$stderr" | sed 's/\: /\:/g' )
IFS=':'
local stderr_parts=( $shrunk_stderr )
IFS="$mem"
# Storing information on the error
error_file="${stderr_parts[0]}"
error_lineno="${stderr_parts[1]}"
error_message=""
for (( i = 3; i <= ${#stderr_parts[#]}; i++ ))
do
error_message="$error_message "${stderr_parts[$i-1]}": "
done
# Removing last ':' (colon character)
error_message="${error_message%:*}"
# Trim
error_message="$( echo "$error_message" | sed -e 's/^[ \t]*//' | sed -e 's/[ \t]*$//' )"
fi
#
# GETTING BACKTRACE:
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #
_backtrace=$( backtrace 2 )
#
# MANAGING THE OUTPUT:
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #
local lineno=""
regex='^([a-z]{1,}) ([0-9]{1,})$'
if [[ $error_lineno =~ $regex ]]
# The error line was found on the log
# (e.g. type 'ff' without quotes wherever)
# --------------------------------------------------------------
then
local row="${BASH_REMATCH[1]}"
lineno="${BASH_REMATCH[2]}"
echo -e "FILE:\t\t${error_file}"
echo -e "${row^^}:\t\t${lineno}\n"
echo -e "ERROR CODE:\t${error_code}"
test -t 1 && tput setf 6 ## white yellow
echo -e "ERROR MESSAGE:\n$error_message"
else
regex="^${error_file}\$|^${error_file}\s+|\s+${error_file}\s+|\s+${error_file}\$"
if [[ "$_backtrace" =~ $regex ]]
# The file was found on the log but not the error line
# (could not reproduce this case so far)
# ------------------------------------------------------
then
echo -e "FILE:\t\t$error_file"
echo -e "ROW:\t\tunknown\n"
echo -e "ERROR CODE:\t${error_code}"
test -t 1 && tput setf 6 ## white yellow
echo -e "ERROR MESSAGE:\n${stderr}"
# Neither the error line nor the error file was found on the log
# (e.g. type 'cp ffd fdf' without quotes wherever)
# ------------------------------------------------------
else
#
# The error file is the first on backtrace list:
# Exploding backtrace on newlines
mem=$IFS
IFS='
'
#
# Substring: I keep only the carriage return
# (others needed only for tabbing purpose)
IFS=${IFS:0:1}
local lines=( $_backtrace )
IFS=$mem
error_file=""
if test -n "${lines[1]}"
then
array=( ${lines[1]} )
for (( i=2; i<${#array[#]}; i++ ))
do
error_file="$error_file ${array[$i]}"
done
# Trim
error_file="$( echo "$error_file" | sed -e 's/^[ \t]*//' | sed -e 's/[ \t]*$//' )"
fi
echo -e "FILE:\t\t$error_file"
echo -e "ROW:\t\tunknown\n"
echo -e "ERROR CODE:\t${error_code}"
test -t 1 && tput setf 6 ## white yellow
if test -n "${stderr}"
then
echo -e "ERROR MESSAGE:\n${stderr}"
else
echo -e "ERROR MESSAGE:\n${error_message}"
fi
fi
fi
#
# PRINTING THE BACKTRACE:
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #
test -t 1 && tput setf 7 ## white bold
echo -e "\n$_backtrace\n"
#
# EXITING:
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #
test -t 1 && tput setf 4 ## red bold
echo "Exiting!"
test -t 1 && tput sgr0 # Reset terminal
exit "$error_code"
}
trap exit_handler EXIT # ! ! ! TRAP EXIT ! ! !
trap exit ERR # ! ! ! TRAP ERR ! ! !
###~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~##
#
# FUNCTION: BACKTRACE
#
###~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~##
function backtrace
{
local _start_from_=0
local params=( "$#" )
if (( "${#params[#]}" >= "1" ))
then
_start_from_="$1"
fi
local i=0
local first=false
while caller $i > /dev/null
do
if test -n "$_start_from_" && (( "$i" + 1 >= "$_start_from_" ))
then
if test "$first" == false
then
echo "BACKTRACE IS:"
first=true
fi
caller $i
fi
let "i=i+1"
done
}
return 0
Example of usage:
file content: trap-test.sh
#!/bin/bash
source 'lib.trap.sh'
echo "doing something wrong now .."
echo "$foo"
exit 0
Running:
bash trap-test.sh
Output:
doing something wrong now ..
(!) EXIT HANDLER:
FILE: trap-test.sh
LINE: 6
ERROR CODE: 1
ERROR MESSAGE:
foo: unassigned variable
BACKTRACE IS:
1 main trap-test.sh
Exiting!
As you can see from the screenshot below, the output is colored and the error message comes in the used language.
An equivalent alternative to "set -e" is
set -o errexit
It makes the meaning of the flag somewhat clearer than just "-e".
Random addition: to temporarily disable the flag, and return to the default (of continuing execution regardless of exit codes), just use
set +e
echo "commands run here returning non-zero exit codes will not cause the entire script to fail"
echo "false returns 1 as an exit code"
false
set -e
This precludes proper error handling mentioned in other responses, but is quick & effective (just like bash).
Inspired by the ideas presented here, I have developed a readable and convenient way to handle errors in bash scripts in my bash boilerplate project.
By simply sourcing the library, you get the following out of the box (i.e. it will halt execution on any error, as if using set -e thanks to a trap on ERR and some bash-fu):
There are some extra features that help handle errors, such as try and catch, or the throw keyword, that allows you to break execution at a point to see the backtrace. Plus, if the terminal supports it, it spits out powerline emojis, colors parts of the output for great readability, and underlines the method that caused the exception in the context of the line of code.
The downside is - it's not portable - the code works in bash, probably >= 4 only (but I'd imagine it could be ported with some effort to bash 3).
The code is separated into multiple files for better handling, but I was inspired by the backtrace idea from the answer above by Luca Borrione.
To read more or take a look at the source, see GitHub:
https://github.com/niieani/bash-oo-framework#error-handling-with-exceptions-and-throw
I prefer something really easy to call. So I use something that looks a little complicated, but is easy to use. I usually just copy-and-paste the code below into my scripts. An explanation follows the code.
#This function is used to cleanly exit any script. It does this displaying a
# given error message, and exiting with an error code.
function error_exit {
echo
echo "$#"
exit 1
}
#Trap the killer signals so that we can exit with a good message.
trap "error_exit 'Received signal SIGHUP'" SIGHUP
trap "error_exit 'Received signal SIGINT'" SIGINT
trap "error_exit 'Received signal SIGTERM'" SIGTERM
#Alias the function so that it will print a message with the following format:
#prog-name(#line#): message
#We have to explicitly allow aliases, we do this because they make calling the
#function much easier (see example).
shopt -s expand_aliases
alias die='error_exit "Error ${0}(#`echo $(( $LINENO - 1 ))`):"'
I usually put a call to the cleanup function in side the error_exit function, but this varies from script to script so I left it out. The traps catch the common terminating signals and make sure everything gets cleaned up. The alias is what does the real magic. I like to check everything for failure. So in general I call programs in an "if !" type statement. By subtracting 1 from the line number the alias will tell me where the failure occurred. It is also dead simple to call, and pretty much idiot proof. Below is an example (just replace /bin/false with whatever you are going to call).
#This is an example useage, it will print out
#Error prog-name (#1): Who knew false is false.
if ! /bin/false ; then
die "Who knew false is false."
fi
Another consideration is the exit code to return. Just "1" is pretty standard, although there are a handful of reserved exit codes that bash itself uses, and that same page argues that user-defined codes should be in the range 64-113 to conform to C/C++ standards.
You might also consider the bit vector approach that mount uses for its exit codes:
0 success
1 incorrect invocation or permissions
2 system error (out of memory, cannot fork, no more loop devices)
4 internal mount bug or missing nfs support in mount
8 user interrupt
16 problems writing or locking /etc/mtab
32 mount failure
64 some mount succeeded
OR-ing the codes together allows your script to signal multiple simultaneous errors.
I use the following trap code, it also allows errors to be traced through pipes and 'time' commands
#!/bin/bash
set -o pipefail # trace ERR through pipes
set -o errtrace # trace ERR through 'time command' and other functions
function error() {
JOB="$0" # job name
LASTLINE="$1" # line of error occurrence
LASTERR="$2" # error code
echo "ERROR in ${JOB} : line ${LASTLINE} with exit code ${LASTERR}"
exit 1
}
trap 'error ${LINENO} ${?}' ERR
I've used
die() {
echo $1
kill $$
}
before; i think because 'exit' was failing for me for some reason. The above defaults seem like a good idea, though.
This has served me well for a while now. It prints error or warning messages in red, one line per parameter, and allows an optional exit code.
# Custom errors
EX_UNKNOWN=1
warning()
{
# Output warning messages
# Color the output red if it's an interactive terminal
# #param $1...: Messages
test -t 1 && tput setf 4
printf '%s\n' "$#" >&2
test -t 1 && tput sgr0 # Reset terminal
true
}
error()
{
# Output error messages with optional exit code
# #param $1...: Messages
# #param $N: Exit code (optional)
messages=( "$#" )
# If the last parameter is a number, it's not part of the messages
last_parameter="${messages[#]: -1}"
if [[ "$last_parameter" =~ ^[0-9]*$ ]]
then
exit_code=$last_parameter
unset messages[$((${#messages[#]} - 1))]
fi
warning "${messages[#]}"
exit ${exit_code:-$EX_UNKNOWN}
}
Not sure if this will be helpful to you, but I modified some of the suggested functions here in order to include the check for the error (exit code from prior command) within it.
On each "check" I also pass as a parameter the "message" of what the error is for logging purposes.
#!/bin/bash
error_exit()
{
if [ "$?" != "0" ]; then
log.sh "$1"
exit 1
fi
}
Now to call it within the same script (or in another one if I use export -f error_exit) I simply write the name of the function and pass a message as parameter, like this:
#!/bin/bash
cd /home/myuser/afolder
error_exit "Unable to switch to folder"
rm *
error_exit "Unable to delete all files"
Using this I was able to create a really robust bash file for some automated process and it will stop in case of errors and notify me (log.sh will do that)
This trick is useful for missing commands or functions. The name of the missing function (or executable) will be passed in $_
function handle_error {
status=$?
last_call=$1
# 127 is 'command not found'
(( status != 127 )) && return
echo "you tried to call $last_call"
return
}
# Trap errors.
trap 'handle_error "$_"' ERR
This function has been serving me rather well recently:
action () {
# Test if the first parameter is non-zero
# and return straight away if so
if test $1 -ne 0
then
return $1
fi
# Discard the control parameter
# and execute the rest
shift 1
"$#"
local status=$?
# Test the exit status of the command run
# and display an error message on failure
if test ${status} -ne 0
then
echo Command \""$#"\" failed >&2
fi
return ${status}
}
You call it by appending 0 or the last return value to the name of the command to run, so you can chain commands without having to check for error values. With this, this statement block:
command1 param1 param2 param3...
command2 param1 param2 param3...
command3 param1 param2 param3...
command4 param1 param2 param3...
command5 param1 param2 param3...
command6 param1 param2 param3...
Becomes this:
action 0 command1 param1 param2 param3...
action $? command2 param1 param2 param3...
action $? command3 param1 param2 param3...
action $? command4 param1 param2 param3...
action $? command5 param1 param2 param3...
action $? command6 param1 param2 param3...
<<<Error-handling code here>>>
If any of the commands fail, the error code is simply passed to the end of the block. I find it useful when you don't want subsequent commands to execute if an earlier one failed, but you also don't want the script to exit straight away (for example, inside a loop).
Sometimes set -e , trap ERR ,set -o ,set -o pipefail and set -o errtrace not work properly because they attempt to add automatic error detection to the shell. This does not work well in practice.
In my opinion, instead of using set -e and other stuffs, you should write your own error checking code. If you wise to use set -e, be aware of potential gotchas.
To avoid Error while running the code you can use exec 1>/dev/null or exec 2>/dev/null
/dev/null in Linux is a null device file. This will discard anything written to it and will return EOF on reading. you can use this at end of the command
For try/catch you can use && or || to achieve Similar behaviour
use can use && like this
{ # try
command &&
# your command
} || {
# catch exception
}
or you can use if else :
if [[ Condition ]]; then
# if true
else
# if false
fi
$? show output of the last command ,it return 1 or 0
Using trap is not always an option. For example, if you're writing some kind of re-usable function that needs error handling and that can be called from any script (after sourcing the file with helper functions), that function cannot assume anything about exit time of the outer script, which makes using traps very difficult. Another disadvantage of using traps is bad composability, as you risk overwriting previous trap that might be set earlier up in the caller chain.
There is a little trick that can be used to do proper error handling without traps. As you may already know from other answers, set -e doesn't work inside commands if you use || operator after them, even if you run them in a subshell; e.g., this wouldn't work:
#!/bin/sh
# prints:
#
# --> outer
# --> inner
# ./so_1.sh: line 16: some_failed_command: command not found
# <-- inner
# <-- outer
set -e
outer() {
echo '--> outer'
(inner) || {
exit_code=$?
echo '--> cleanup'
return $exit_code
}
echo '<-- outer'
}
inner() {
set -e
echo '--> inner'
some_failed_command
echo '<-- inner'
}
outer
But || operator is needed to prevent returning from the outer function before cleanup. The trick is to run the inner command in background, and then immediately wait for it. The wait builtin will return the exit code of the inner command, and now you're using || after wait, not the inner function, so set -e works properly inside the latter:
#!/bin/sh
# prints:
#
# --> outer
# --> inner
# ./so_2.sh: line 27: some_failed_command: command not found
# --> cleanup
set -e
outer() {
echo '--> outer'
inner &
wait $! || {
exit_code=$?
echo '--> cleanup'
return $exit_code
}
echo '<-- outer'
}
inner() {
set -e
echo '--> inner'
some_failed_command
echo '<-- inner'
}
outer
Here is the generic function that builds upon this idea. It should work in all POSIX-compatible shells if you remove local keywords, i.e. replace all local x=y with just x=y:
# [CLEANUP=cleanup_cmd] run cmd [args...]
#
# `cmd` and `args...` A command to run and its arguments.
#
# `cleanup_cmd` A command that is called after cmd has exited,
# and gets passed the same arguments as cmd. Additionally, the
# following environment variables are available to that command:
#
# - `RUN_CMD` contains the `cmd` that was passed to `run`;
# - `RUN_EXIT_CODE` contains the exit code of the command.
#
# If `cleanup_cmd` is set, `run` will return the exit code of that
# command. Otherwise, it will return the exit code of `cmd`.
#
run() {
local cmd="$1"; shift
local exit_code=0
local e_was_set=1; if ! is_shell_attribute_set e; then
set -e
e_was_set=0
fi
"$cmd" "$#" &
wait $! || {
exit_code=$?
}
if [ "$e_was_set" = 0 ] && is_shell_attribute_set e; then
set +e
fi
if [ -n "$CLEANUP" ]; then
RUN_CMD="$cmd" RUN_EXIT_CODE="$exit_code" "$CLEANUP" "$#"
return $?
fi
return $exit_code
}
is_shell_attribute_set() { # attribute, like "x"
case "$-" in
*"$1"*) return 0 ;;
*) return 1 ;;
esac
}
Example of usage:
#!/bin/sh
set -e
# Source the file with the definition of `run` (previous code snippet).
# Alternatively, you may paste that code directly here and comment the next line.
. ./utils.sh
main() {
echo "--> main: $#"
CLEANUP=cleanup run inner "$#"
echo "<-- main"
}
inner() {
echo "--> inner: $#"
sleep 0.5; if [ "$1" = 'fail' ]; then
oh_my_god_look_at_this
fi
echo "<-- inner"
}
cleanup() {
echo "--> cleanup: $#"
echo " RUN_CMD = '$RUN_CMD'"
echo " RUN_EXIT_CODE = $RUN_EXIT_CODE"
sleep 0.3
echo '<-- cleanup'
return $RUN_EXIT_CODE
}
main "$#"
Running the example:
$ ./so_3 fail; echo "exit code: $?"
--> main: fail
--> inner: fail
./so_3: line 15: oh_my_god_look_at_this: command not found
--> cleanup: fail
RUN_CMD = 'inner'
RUN_EXIT_CODE = 127
<-- cleanup
exit code: 127
$ ./so_3 pass; echo "exit code: $?"
--> main: pass
--> inner: pass
<-- inner
--> cleanup: pass
RUN_CMD = 'inner'
RUN_EXIT_CODE = 0
<-- cleanup
<-- main
exit code: 0
The only thing that you need to be aware of when using this method is that all modifications of Shell variables done from the command you pass to run will not propagate to the calling function, because the command runs in a subshell.

In bash, either exit script without exiting the shell or export/set variables from within subshell

I have a function that runs a set of scripts that set variables, functions, and aliases in the current shell.
reloadVariablesFromScript() {
for script in "${scripts[#]}"; do
. "$script"
done
}
If one of the scripts has an error, I want to exit the script and then exit the function, but not to kill the shell.
reloadVariablesFromScript() {
for script in "${scripts[#]}"; do
{(
set -e
. "$script"
)}
if [[ $? -ne 0 ]]; then
>&2 echo $script failed. Skipping remaining scripts.
return 1
fi
done
}
This would do what I want except it doesn't set the variables in the script whether the script succeeds or fails.
Without the subshell, set -e causes the whole shell to exit, which is undesirable.
Is there a way I can either prevent the called script from continuing on an error without killing the shell or else set/export variables, aliases, and functions from within a subshell?
The following script simulates my problem:
test() {
{(
set -e
export foo=bar
false
echo Should not have gotten here!
export bar=baz
)}
local errorCode=$?
echo foo="'$foo'". It should equal 'bar'.
echo bar="'$bar'". It should not be set.
if [[ $errorCode -ne 0 ]]; then
echo Script failed correctly. Exiting function.
return 1
fi
echo Should not have gotten here!
}
test
If worst comes to worse, since these scripts don't actually edit the filesystem, I can run each script in a subshell, check the exit code, and if it succeeds, run it outside of a subshell.
Note that set -e has a number of surprising behaviors -- relying on it is not universally considered a good idea. That caveat being give, though: We can shuffle environment variables, aliases, and shell functions out as text:
envTest() {
local errorCode newVars
newVars=$(
set -e
{
export foo=bar
false
echo Should not have gotten here!
export bar=baz
} >&2
# print generate code which, when eval'd, recreates our functions and variables
declare -p | egrep -v '^declare -[^[:space:]]*r'
declare -f
alias -p
); errorCode=$?
if (( errorCode == 0 )); then
eval "$newVars"
fi
printf 'foo=%q. It should equal %q\n' "$foo" "bar"
printf 'bar=%q. It should not be set.\n' "$bar"
if [[ $errorCode -ne 0 ]]; then
echo 'Script failed correctly. Exiting function.'
return 1
fi
echo 'Should not have gotten here!'
}
envTest
Note that this code only evaluates either export should the entire script segment succeed; the question text and comments appear to indicate that this is acceptable if not desired.

bash script syntax error at function call and passing array as argument

I'm new to bash so assume that I don't understand everything in this simple script as I've been putting this together as of today with no prior experience with bash.
I get this error when I run test.sh:
command substitution: line 29: syntax error near unexpected token `$1,'
./f.sh: command substitution: line 29: `index_of($1, $urls))'
FILE: f.sh
#!/bin/bash
urls=( "example.com" "example2.com")
error_exit()
{
echo "$1" 1>&2
exit 1
}
index_of(){
needle=$1
haystack=$2
for i in "${!haystack[#]}"; do
if [[ "${haystack[$i]}" = "${needle}" ]]; then
echo "${i}"
fi
done
echo -1
}
validate_url_param(){
index=-2 #-2 as flag
if [ $# -eq 0 ]; then
error_exit "No url provided. Exiting"
else
index=$(index_of($1, $urls)) #ERROR POINTS TO THIS LINE
if [ $index -eq -1 ]; then
error_exit "Provided url not found in list. Exiting"
fi
fi
echo $index
}
FILE: test.sh
#!/bin/bash
. ./f.sh
index=$(validate_url_param "example.com")
echo $index
echo "${urls[0]}"
I've lost track of all of the tweaks I tried but google is failing me and I'm sure this is some basic stuff so... thanks in advance.
The immediate error, just like the error message tells you, is that shell functions (just like shell scripts) do not require or accept commas between their arguments or parentheses around the argument list. But there are several changes you could make to improve this code.
Here's a refactored version, with inlined comments.
#!/bin/bash
urls=("example.com" "example2.com")
error_exit()
{
# Include script name in error message; echo all parameters
echo "$0: $#" 1>&2
exit 1
}
# A function can't really accept an array. But it's easy to fix:
# make the first argument the needle, and the rest, the haystack.
# Also, mark variables as local
index_of(){
local needle=$1
shift
local i
for ((i=1; i<=$#; ++i)); do
if [[ "${!i}" = "${needle}" ]]; then
echo "${i}"
# Return when you found it
return 0
fi
done
# Don't echo anything on failure; just return false
return 1
}
validate_url_param(){
# global ${urls[#]} is still a bit of a wart
if [ $# -eq 0 ]; then
error_exit "No url provided. Exiting"
else
if ! index_of "$1" "${urls[#]}"; then
error_exit "Provided url not found in list. Exiting"
fi
fi
}
# Just run the function from within the script itself
validate_url_param "example.com"
echo "${urls[0]}"
Notice how the validate_url_param function doesn't capture the output from the function it is calling. index_of simply prints the result to standard output and that's fine, just let that happen and don't intervene. The exit code tells us whether it succeeded or not.
However, reading the URLs into memory is often not useful or necessary. Perhaps you are simply looking for
grep -Fx example.com urls.txt

Check if file exists [BASH]

How do I check if file exists in bash?
When I try to do it like this:
FILE1="${#:$OPTIND:1}"
if [ ! -e "$FILE1" ]
then
echo "requested file doesn't exist" >&2
exit 1
elif
<more code follows>
I always get following output:
requested file doesn't exist
The program is used like this:
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
Any ideas please?
I will be glad for any help.
P.S. I wish I could show the entire file without the risk of being fired from school for having a duplicate. If there is a private method of communication I will happily oblige.
My mistake. Fas forcing a binary file into a wrong place. Thanks for everyone's help.
Little trick to debugging problems like this. Add these lines to the top of your script:
export PS4="\$LINENO: "
set -xv
The set -xv will print out each line before it is executed, and then the line once the shell interpolates variables, etc. The $PS4 is the prompt used by set -xv. This will print the line number of the shell script as it executes. You'll be able to follow what is going on and where you may have problems.
Here's an example of a test script:
#! /bin/bash
export PS4="\$LINENO: "
set -xv
FILE1="${#:$OPTIND:1}" # Line 6
if [ ! -e "$FILE1" ] # Line 7
then
echo "requested file doesn't exist" >&2
exit 1
else
echo "Found File $FILE1" # Line 12
fi
And here's what I get when I run it:
$ ./test.sh .profile
FILE1="${#:$OPTIND:1}"
6: FILE1=.profile
if [ ! -e "$FILE1" ]
then
echo "requested file doesn't exist" >&2
exit 1
else
echo "Found File $FILE1"
fi
7: [ ! -e .profile ]
12: echo 'Found File .profile'
Found File .profile
Here, I can see that I set $FILE1 to .profile, and that my script understood that ${#:$OPTIND:1}. The best thing about this is that it works on all shells down to the original Bourne shell. That means if you aren't running Bash as you think you might be, you'll see where your script is failing, and maybe fix the issue.
I suspect you might not be running your script in Bash. Did you put #! /bin/bash on the top?
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
You may want to use getopts to parse your parameters:
#! /bin/bash
USAGE=" Usage:
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
"
while getopts gpr:d: option
do
case $option in
g) g_opt=1;;
p) p_opt=1;;
r) rfunction_id="$OPTARG";;
d) dfunction_id="$OPTARG";;
[?])
echo "Invalid Usage" 1>&2
echo "$USAGE" 1>&2
exit 2
;;
esac
done
if [[ -n $rfunction_id && -n $dfunction_id ]]
then
echo "Invalid Usage: You can't specify both -r and -d" 1>&2
echo "$USAGE" >2&
exit 2
fi
shift $(($OPTIND - 1))
[[ -n $g_opt ]] && echo "-g was set"
[[ -n $p_opt ]] && echo "-p was set"
[[ -n $rfunction_id ]] && echo "-r was set to $rfunction_id"
[[ -n $dfunction_id ]] && echo "-d was set to $dfunction_id"
[[ -n $1 ]] && echo "File is $1"
To (recap) and add to #DavidW.'s excellent answer:
Check the shebang line (first line) of your script to ensure that it's executed by bash: is it #!/bin/bash or #!/usr/bin/env bash?
Inspect your script file for hidden control characters (such as \r) that can result in unexpected behavior; run cat -v scriptFile | fgrep ^ - it should produce NO output; if the file does contain \r chars., they would show as ^M.
To remove the \r instances (more accurately, to convert Windows-style \r\n newline sequences to Unix \n-only sequences), you can use dos2unix file to convert in place; if you don't have this utility, you can use sed 's/'$'\r''$//' file > outfile (CAVEAT: use a DIFFERENT output file, otherwise you'll destroy your input file); to remove all \r instances (even if not followed by \n), use tr -d '\r' < file > outfile (CAVEAT: use a DIFFERENT output file, otherwise you'll destroy your input file).
In addition to #DavidW.'s great debugging technique, you can add the following to visually inspect all arguments passed to your script:
i=0; for a; do echo "\$$((i+=1))=[$a]"; done
(The purpose of enclosing the value in [...] (for example), is to see the exact boundaries of the values.)
This will yield something like:
$1=[-g]
$2=[input.txt]
...
Note, though, that nothing at all is printed if no arguments were passed.
Try to print FILE1 to see if it has the value you want, if it is not the problem, here is a simple script (site below):
#!/bin/bash
file="${#:$OPTIND:1}"
if [ -f "$file" ]
then
echo "$file found."
else
echo "$file not found."
fi
http://www.cyberciti.biz/faq/unix-linux-test-existence-of-file-in-bash/
Instead of plucking an item out of "$#" in a tricky way, why don't you shift off the args you've processed with getopts:
while getopts ...
done
shift $(( OPTIND - 1 ))
FILE1=$1

Equivalent of __FILE__ and __LINE__ in Bash

Is there any variable in bash that contains the name of the .sh file executed? The line number would be great too.
I want to use it in error messages such as:
echo "ERROR: [$FILE:L$LINE] $somefile not found"
#!/bin/bash
echo $LINENO
echo `basename $0`
$LINENO for the current line number
$0 for the current file. I used basename to ensure you only get the file name and not the path.
UPDATE:
#!/bin/bash
MY_NAME=`basename $0`
function ouch {
echo "Fail # [${MY_NAME}:${1}]"
exit 1
}
ouch $LINENO
You have to pass the line as a parameter if you use the function approach else you will get the line of the function definition.
I find the "BASH_SOURCE" and "BASH_LINENO" built-in arrays very useful:
$ cat xx
#!/bin/bash
_ERR_HDR_FMT="%.23s %s[%s]: "
_ERR_MSG_FMT="${_ERR_HDR_FMT}%s\n"
error_msg() {
printf "$_ERR_MSG_FMT" $(date +%F.%T.%N) ${BASH_SOURCE[1]##*/} ${BASH_LINENO[0]} "${#}"
}
error_msg "here"
error_msg "and here"
Invoking xx yields
2010-06-16.15:33:13.069 xx[11]: here
2010-06-16.15:33:13.073 xx[14]: and here
You just need to
echo $LINENO
echo $(basename $0)
Here's how to do it in a reusable function. if the following is in a file named script:
#!/bin/bash
debug() {
echo "${BASH_SOURCE[1]##*/}:${FUNCNAME[1]}[${BASH_LINENO[0]}]" > /dev/tty
}
debug
This produces the output:
script:main[5]
Which indicates the line on which debug was called.
The following will print out the filename, function, line and an optional message.
Also works in zsh for extra goodness.
# Say the file, line number and optional message for debugging
# Inspired by bash's `caller` builtin
# Thanks to https://unix.stackexchange.com/a/453153/143394
function yelp () {
# shellcheck disable=SC2154 # undeclared zsh variables in bash
if [[ $BASH_VERSION ]]; then
local file=${BASH_SOURCE[1]##*/} func=${FUNCNAME[1]} line=${BASH_LINENO[0]}
else # zsh
emulate -L zsh # because we may be sourced by zsh `emulate bash -c`
# $funcfiletrace has format: file:line
local file=${funcfiletrace[1]%:*} line=${funcfiletrace[1]##*:}
local func=${funcstack[2]}
[[ $func =~ / ]] && func=source # $func may be filename. Use bash behaviour
fi
echo "${file##*/}:$func:$line $*" > /dev/tty
}

Resources