I want to run this command source .env (sourcing a .env file) and if the .env file had some errors while sourcing. I want to show a message before the error output "Hey you got errors in your .env" else if there's no error, I don't want to show anything.
Here's a code sample that needs editing:
#!/bin/zsh
env_auto_sourcing() {
if [[ -f .env ]]; then
OUTPUT="$(source .env &> /dev/null)"
echo "${OUTPUT}"
if [ -n "$OUTPUT" ]; then
echo "Hey you got errors in your .env"
echo "$OUTPUT"
fi
}
You could use bash -n (zsh has has a -n option as well) to syntax check your script before sourcing it:
env_auto_sourcing() {
if [[ -f .env ]]; then
if errs=$(bash -n .env 2>&1);
then source .env;
else
printf '%s\n' "Hey you got errors" "$errs";
fi
fi
}
Storing the syntax check errors in a file is a little cleaner than the subshell approach you have used in your code.
bash -n has a few pitfalls as seen here:
How do I check syntax in bash without running the script?
Why not just use the exit code from the command source ?
You don't have to use bash -n for this because ...
If let's say your .env file contains these 2 invalid lines:
dsadsd
sdss
If you run your current accepted code using the example above:
if errs=$(bash -n .env 2>&1);
the above condition will fail to stop the file from sourcing.
So, you can use source command return code to handle all of this:
#!/bin/bash
# This doesn't actually source it. It just test if source is working
errs=$(source ".env" 2>&1 >/dev/null)
# get the return code
retval=$?
#echo "${retval}"
if [ ${retval} = 0 ]; then
# Do another check for any syntax error
if [ -n "${errs}" ]; then
echo "The source file returns 0 but you got syntax errors: "
echo "Error details:"
printf "%s\n" "${errs}"
exit 1
else
# Success here. we know that this command works without error so we source it
echo "The source file returns 0 and no syntax errors: "
source ".env"
fi
else
echo "The source command returns an error code ${retval}: "
echo "Error details:"
printf "%s\n" "${errs}"
exit 1
fi
The best thing with this approach is, it will check both bash syntax and source syntax as well:
Now you can test this data in your env file:
-
~
#
~<
>
Related
I want to create a utility function for bash to remove duplicate lines. I am using function
function remove_empty_lines() {
if ! command -v awk &> /dev/null
then
echo '[x] ERR: "awk" command not found'
return
fi
if [[ -z "$1" ]]
then
echo "usage: remove_empty_lines <file-name> [--replace]"
echo
echo "Arguments:"
echo -e "\t--replace\t (Optional) If not passed, the result will be redirected to stdout"
return
fi
if [[ ! -f "$1" ]]
then
echo "[x] ERR: \"$1\" file not found"
return
fi
echo $0
local CMD="awk '!seen[$0]++' $1"
if [[ "$2" = '--reload' ]]
then
CMD+=" > $1"
fi
echo $CMD
}
If I am running the main awk command directly, it is working. But when i execute the same $CMD in the function, I am getting this error
$ remove_empty_lines app.js
/bin/bash
awk '!x[/bin/bash]++' app.js
The original code is broken in several ways:
When used with --reload, it would truncate the output file's contents before awk could ever read those contents (see How can I use a file in a command and redirect output to the same file without truncating it?)
It didn't ever actually run the command, and for the reasons described in BashFAQ #50, storing a shell command in a string is inherently buggy (one can work around some of those issues with eval; BashFAQ #48 describes why doing so introduces security bugs).
It wrote error messages (and other "diagnostic content") to stdout instead of stderr; this means that if your function's output was redirected to a file, you could never see its errors -- they'd end up jumbled into the output.
Error cases were handled with a return even in cases where $? would be zero; this means that return itself would return a zero/successful/truthy status, not revealing to the caller that any error had taken place.
Presumably the reason you were storing your output in CMD was to be able to perform a redirection conditionally, but that can be done other ways: Below, we always create a file descriptor out_fd, but point it to either stdout (when called without --reload), or to a temporary file (if called with --reload); if-and-only-if awk succeeds, we then move the temporary file over the output file, thus replacing it as an atomic operation.
remove_empty_lines() {
local out_fd rc=0 tempfile=
command -v awk &>/dev/null || { echo '[x] ERR: "awk" command not found' >&2; return 1; }
if [[ -z "$1" ]]; then
printf '%b\n' >&2 \
'usage: remove_empty_lines <file-name> [--replace]' \
'' \
'Arguments:' \
'\t--replace\t(Optional) If not passed, the result will be redirected to stdout'
return 1
fi
[[ -f "$1" ]] || { echo "[x] ERR: \"$1\" file not found" >&2; return 1; }
if [ "$2" = --reload ]; then
tempfile=$(mktemp -t "$1.XXXXXX") || return
exec {out_fd}>"$tempfile" || { rc=$?; rm -f "$tempfile"; return "$rc"; }
else
exec {out_fd}>&1
fi
awk '!seen[$0]++' <"$1" >&$out_fd || { rc=$?; rm -f "$tempfile"; return "$rc"; }
exec {out_fd}>&- # close our file descriptor
if [[ $tempfile ]]; then
mv -- "$tempfile" "$1" || return
fi
}
First off the output from your function call is not an error but rather the output of two echo commands (echo $0 and echo $CMD).
And as Charles Duffy has pointed out ... at no point is the function actually running the $CMD.
As for the inclusion of /bin/bash in your function's echo output ... the main problem is the reference to $0; by definition $0 is the name of the running process, which in the case of a function is the shell under which the function is being called. Consider the following when run from a bash command prompt:
$ echo $0
-bash
As you can see from your output this generates /bin/bash in your environment. See this and this for more details.
On a related note, the reference to $0 within double quotes causes the $0 to be evaluated, so this:
local CMD="awk '!seen[$0]++' $1"
becomes
local CMD="awk '!seen[/bin/bash]++' app.js"
I'm thinking what you want is something like:
echo $1 # the name of the file to be processed
local CMD="awk '!seen[\$0]++' $1" # escape the '$' in '$0'
becomes
local CMD="awk '!seen[$0]++' app.js"
That should fix the issues shown in your function's output; as for the other issues ... you're getting a good bit of feedback in the various comments ...
I'm new to bash so assume that I don't understand everything in this simple script as I've been putting this together as of today with no prior experience with bash.
I get this error when I run test.sh:
command substitution: line 29: syntax error near unexpected token `$1,'
./f.sh: command substitution: line 29: `index_of($1, $urls))'
FILE: f.sh
#!/bin/bash
urls=( "example.com" "example2.com")
error_exit()
{
echo "$1" 1>&2
exit 1
}
index_of(){
needle=$1
haystack=$2
for i in "${!haystack[#]}"; do
if [[ "${haystack[$i]}" = "${needle}" ]]; then
echo "${i}"
fi
done
echo -1
}
validate_url_param(){
index=-2 #-2 as flag
if [ $# -eq 0 ]; then
error_exit "No url provided. Exiting"
else
index=$(index_of($1, $urls)) #ERROR POINTS TO THIS LINE
if [ $index -eq -1 ]; then
error_exit "Provided url not found in list. Exiting"
fi
fi
echo $index
}
FILE: test.sh
#!/bin/bash
. ./f.sh
index=$(validate_url_param "example.com")
echo $index
echo "${urls[0]}"
I've lost track of all of the tweaks I tried but google is failing me and I'm sure this is some basic stuff so... thanks in advance.
The immediate error, just like the error message tells you, is that shell functions (just like shell scripts) do not require or accept commas between their arguments or parentheses around the argument list. But there are several changes you could make to improve this code.
Here's a refactored version, with inlined comments.
#!/bin/bash
urls=("example.com" "example2.com")
error_exit()
{
# Include script name in error message; echo all parameters
echo "$0: $#" 1>&2
exit 1
}
# A function can't really accept an array. But it's easy to fix:
# make the first argument the needle, and the rest, the haystack.
# Also, mark variables as local
index_of(){
local needle=$1
shift
local i
for ((i=1; i<=$#; ++i)); do
if [[ "${!i}" = "${needle}" ]]; then
echo "${i}"
# Return when you found it
return 0
fi
done
# Don't echo anything on failure; just return false
return 1
}
validate_url_param(){
# global ${urls[#]} is still a bit of a wart
if [ $# -eq 0 ]; then
error_exit "No url provided. Exiting"
else
if ! index_of "$1" "${urls[#]}"; then
error_exit "Provided url not found in list. Exiting"
fi
fi
}
# Just run the function from within the script itself
validate_url_param "example.com"
echo "${urls[0]}"
Notice how the validate_url_param function doesn't capture the output from the function it is calling. index_of simply prints the result to standard output and that's fine, just let that happen and don't intervene. The exit code tells us whether it succeeded or not.
However, reading the URLs into memory is often not useful or necessary. Perhaps you are simply looking for
grep -Fx example.com urls.txt
I am working on the shell script to read config properties from a .properties files, below is the sample config
RCTP_servername=test1
RCTP_databasename=test2
RCTP_portnumber=test3
RCTP_username=test4
RCTP_password=test5
i have written a shell script as below but it doesn't work could anyone please have look and guide me how to solve this
#building the keys based on environment
environment=RCTP
servername_key="$environment"_servername
databasename_key="$environment"_databasename
portnumber_key="$environment"_portnumber
username_key="$environment"_username
password_key="$environment"_username
#read the config.properties files
file=serverconfig.properties
if [ -f "$file" ]
then
echo "$file found."
while IFS='=' read -r key value
do
key=$(echo $key )
eval "${key}='${value}'"
done < "$file"
servername_value=${servername_key}
databasename_value=${databasename_key}
portnumber_value=${portnumber_key}
username_value=${username_key}
password_value=${password_key}
else
echo "$file not found."
fi
echo "$servername_value"
but am getting an below when i tried to run it, the error is ./test_script_fte.sh: line 23: ${servername_key}: bad substitution
The expected output is when echo $servername_value executed is test1
Though eval is not recommended most of the time, here is a solution that uses indirect-reference as
echo "${!servername_value}"
I have also tweaked the logic to source the properties file over using eval from your logic. With the complete script as below.
#!/bin/bash
#building the keys based on environment
environment=RCTP
servername_key="$environment"_servername
databasename_key="$environment"_databasename
portnumber_key="$environment"_portnumber
username_key="$environment"_username
password_key="$environment"_username
#read the config.properties files
file=serverconfig.properties
if [ -f "$file" ]
then
echo "$file found."
# sourcing the properties file in the current shell to fetch the values
source "$file"
servername_value=${servername_key}
databasename_value=${databasename_key}
portnumber_value=${portnumber_key}
username_value=${username_key}
password_value=${password_key}
else
echo "$file not found."
fi
echo "${!servername_value}"
echo "${!databasename_value}"
echo "${!portnumber_value}"
echo "${!username_value}"
echo "${!password_value}"
It seems you want to use the value of a variable as the name of another variable.
please replace your last line with followings
eval echo \"\$$servername_value\"
Trying to debug my bash script. What's wrong with my syntax here? I'm trying to evaluate a parameter entered by the user and based on that run one of my IF-THEN statements. However, I'm getting a command not found.
Here's my script thus far:
if [[ $# != 4 ]]; then
echo "Usage: ./test.sh <ABC|XYZ> <owner> <db> <TARGETHOST>" 2>&1
exit 1
fi
case $1 in
ABC|XYZ)
filename="get-$1.sql"
;;
*)echo "Must enter ABC or XYZ"
exit 1
;;
esac
export OWNER=$2
export DB=$3
export HOST_NM=$4
export PORT=5432
export LOG="test-$1.log"
PSQL=`which psql`
if [[$1=="ABC"]]; then
RUNCLI=$("$PSQL" -h $HOST_NM -p $PORT -U $OWNER $DB -F $'\t' --no-align -f get-$1.sql | tee >> $LOG)
exit 1
else
echo "Error running report ..."
fi
if [[$1=="XYZ"]]; then
RUNCLI2=$("$PSQL" -h $HOST_NM -p $PORT -U $OWNER $DB -a -f get-$1.sql | tee >> $LOG)
exit 1
else
echo "Error running report ..."
fi
Error:
./test.sh: line 41: [[XYZ==ABC]]: command not found
Error running report ...
./test.sh: line 51: [[XYZ==XYZ]]: command not found
Error running report ...
Although the question is already answered in the comment section I want to give an answer and share some knowledge which is not obvious (at least it was not for me).
The if in bash just checks the return code of the following command, which means that instead of if [ condition ]... or if [[ condition ]]... you could also write if ./configure && make....
[ and [[ are commands or shell-built-ins, respectively, as well and not part of the if syntax. An which [ for instance returns /bin/[.
At this point it is obvious that you need spaces between the brackets and the condition since it is just just a set of parameters passed to a command.
If you have this in mind, you will never forget the spaces again.
How do I check if file exists in bash?
When I try to do it like this:
FILE1="${#:$OPTIND:1}"
if [ ! -e "$FILE1" ]
then
echo "requested file doesn't exist" >&2
exit 1
elif
<more code follows>
I always get following output:
requested file doesn't exist
The program is used like this:
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
Any ideas please?
I will be glad for any help.
P.S. I wish I could show the entire file without the risk of being fired from school for having a duplicate. If there is a private method of communication I will happily oblige.
My mistake. Fas forcing a binary file into a wrong place. Thanks for everyone's help.
Little trick to debugging problems like this. Add these lines to the top of your script:
export PS4="\$LINENO: "
set -xv
The set -xv will print out each line before it is executed, and then the line once the shell interpolates variables, etc. The $PS4 is the prompt used by set -xv. This will print the line number of the shell script as it executes. You'll be able to follow what is going on and where you may have problems.
Here's an example of a test script:
#! /bin/bash
export PS4="\$LINENO: "
set -xv
FILE1="${#:$OPTIND:1}" # Line 6
if [ ! -e "$FILE1" ] # Line 7
then
echo "requested file doesn't exist" >&2
exit 1
else
echo "Found File $FILE1" # Line 12
fi
And here's what I get when I run it:
$ ./test.sh .profile
FILE1="${#:$OPTIND:1}"
6: FILE1=.profile
if [ ! -e "$FILE1" ]
then
echo "requested file doesn't exist" >&2
exit 1
else
echo "Found File $FILE1"
fi
7: [ ! -e .profile ]
12: echo 'Found File .profile'
Found File .profile
Here, I can see that I set $FILE1 to .profile, and that my script understood that ${#:$OPTIND:1}. The best thing about this is that it works on all shells down to the original Bourne shell. That means if you aren't running Bash as you think you might be, you'll see where your script is failing, and maybe fix the issue.
I suspect you might not be running your script in Bash. Did you put #! /bin/bash on the top?
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
You may want to use getopts to parse your parameters:
#! /bin/bash
USAGE=" Usage:
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
"
while getopts gpr:d: option
do
case $option in
g) g_opt=1;;
p) p_opt=1;;
r) rfunction_id="$OPTARG";;
d) dfunction_id="$OPTARG";;
[?])
echo "Invalid Usage" 1>&2
echo "$USAGE" 1>&2
exit 2
;;
esac
done
if [[ -n $rfunction_id && -n $dfunction_id ]]
then
echo "Invalid Usage: You can't specify both -r and -d" 1>&2
echo "$USAGE" >2&
exit 2
fi
shift $(($OPTIND - 1))
[[ -n $g_opt ]] && echo "-g was set"
[[ -n $p_opt ]] && echo "-p was set"
[[ -n $rfunction_id ]] && echo "-r was set to $rfunction_id"
[[ -n $dfunction_id ]] && echo "-d was set to $dfunction_id"
[[ -n $1 ]] && echo "File is $1"
To (recap) and add to #DavidW.'s excellent answer:
Check the shebang line (first line) of your script to ensure that it's executed by bash: is it #!/bin/bash or #!/usr/bin/env bash?
Inspect your script file for hidden control characters (such as \r) that can result in unexpected behavior; run cat -v scriptFile | fgrep ^ - it should produce NO output; if the file does contain \r chars., they would show as ^M.
To remove the \r instances (more accurately, to convert Windows-style \r\n newline sequences to Unix \n-only sequences), you can use dos2unix file to convert in place; if you don't have this utility, you can use sed 's/'$'\r''$//' file > outfile (CAVEAT: use a DIFFERENT output file, otherwise you'll destroy your input file); to remove all \r instances (even if not followed by \n), use tr -d '\r' < file > outfile (CAVEAT: use a DIFFERENT output file, otherwise you'll destroy your input file).
In addition to #DavidW.'s great debugging technique, you can add the following to visually inspect all arguments passed to your script:
i=0; for a; do echo "\$$((i+=1))=[$a]"; done
(The purpose of enclosing the value in [...] (for example), is to see the exact boundaries of the values.)
This will yield something like:
$1=[-g]
$2=[input.txt]
...
Note, though, that nothing at all is printed if no arguments were passed.
Try to print FILE1 to see if it has the value you want, if it is not the problem, here is a simple script (site below):
#!/bin/bash
file="${#:$OPTIND:1}"
if [ -f "$file" ]
then
echo "$file found."
else
echo "$file not found."
fi
http://www.cyberciti.biz/faq/unix-linux-test-existence-of-file-in-bash/
Instead of plucking an item out of "$#" in a tricky way, why don't you shift off the args you've processed with getopts:
while getopts ...
done
shift $(( OPTIND - 1 ))
FILE1=$1